Data classification method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to a data classification method and device, electronic equipment and storage medium, and relates to the field of insurance and finance technology. The present disclosure solves the problem of low accuracy of data classification results caused by large differences in data volume of positive data and negative data. The method comprises: determining a confidence interval according to the confidence of a plurality of target positive historical data; the confidence represents the correlation degree of the position distribution of each target positive historical data and other target positive historical data in the plurality of target positive historical data; the confidence interval represents the distribution interval of the target positive historical data with a confidence higher than a preset confidence; and classifying the data to be classified according to the confidence interval to determine abnormal data and / or non-abnormal data in the data to be classified.

Description

Technical Field

[0001] This disclosure relates to the field of insurance and financial technology, and in particular to data classification methods, devices, electronic equipment and storage media. Background Technology

[0002] With the development of computer technology, in order to facilitate the management and application of data, it has become a trend to use machine learning models to classify different types of data (such as data of different customers in the insurance industry).

[0003] In related technologies, historical positive and negative sample data are typically used to train a data classification model. This trained model is then used to classify the input data to determine its data type. However, in this method, if the distribution of positive and negative sample data in the historical data is uneven, the classification results may exhibit classification bias. That is, the classification results may favor the data type with the larger sample size, leading to lower accuracy. Summary of the Invention

[0004] This invention provides a data classification method, apparatus, electronic device, and storage medium to at least solve the problem in related technologies where the large difference in the amount of positive and negative data leads to low accuracy in data classification results. The technical solution of this invention is as follows:

[0005] According to a first aspect of the present invention, a data classification method is provided, the method comprising: determining a confidence interval based on the confidence levels of multiple target positive historical data; the confidence level characterizing the degree of correlation between the positional distribution of each target positive historical data and other target positive historical data in the multiple target positive historical data; the confidence interval characterizing the distribution range of target positive historical data with a confidence level higher than a preset confidence level; and classifying the data to be classified according to the confidence interval to determine abnormal data and / or non-abnormal data in the data to be classified.

[0006] Furthermore, based on the confidence levels of multiple target positive historical data, a confidence interval is determined, including: identifying the associated data corresponding to each target positive historical data from multiple target positive historical data; the associated data being the first preset number of data with the smallest positional distribution distance difference from the target positive historical data among other target positive historical data; using the average result of the positional distribution distance difference between each target positive historical data and its corresponding associated data as the confidence level of each target positive historical data; and determining the confidence interval based on the confidence levels of each target positive historical data.

[0007] Furthermore, based on the confidence level of each target's positive historical data, a confidence interval is determined, including: determining the confidence interval by the average result of the target confidence levels extracted from the confidence levels of each target's positive historical data based on a preset number of extractions; the number of confidence levels extracted from the confidence levels of each target's positive historical data each time in the preset number of extractions is a second preset number.

[0008] Furthermore, the data to be classified is classified according to the confidence interval to identify outlier and / or non-outlier data in the data to be classified, including: determining the candidate confidence level of each candidate data in the data to be classified; the candidate confidence level characterizes the degree of correlation between each candidate data and other candidate data in the location distribution of the data to be classified; classifying the candidate confidence level of each candidate data according to the confidence interval to obtain the classification result; and identifying outlier and / or non-outlier data in the data to be classified based on the classification result.

[0009] Furthermore, based on the classification results, abnormal and / or non-abnormal data in the data to be classified are determined, including: when the classification results indicate that the candidate confidence level of the candidate data belongs to the confidence interval, the candidate data corresponding to the candidate confidence interval that belongs to the confidence interval is determined as non-abnormal data; when the classification results indicate that the candidate confidence level of the candidate data is outside the confidence interval, the candidate data corresponding to the candidate confidence interval outside the confidence interval is determined as abnormal data.

[0010] Furthermore, the method also includes: obtaining multiple positive historical data from a historical data sample set; and performing dimensionality reduction processing on the multiple positive historical data based on non-negative matrix factorization to obtain multiple target positive historical data.

[0011] Furthermore, the method also includes: obtaining multiple positive historical data from a historical data sample set; normalizing the multiple positive historical data to obtain normalized historical data, wherein the normalized historical data are all within a preset range; and obtaining multiple target positive historical data based on the normalized historical data.

[0012] According to a second aspect of the present invention, a data classification apparatus is provided, the apparatus comprising: a determining unit configured to determine a confidence interval based on the confidence levels of a plurality of target positive historical data; the confidence level characterizes the degree of correlation between the positional distribution of each target positive historical data and other target positive historical data in the plurality of target positive historical data; the confidence interval characterizes the distribution range of target positive historical data with a confidence level higher than a preset confidence level; and a classification unit configured to classify data to be classified based on the confidence interval to determine abnormal data and / or non-abnormal data in the data to be classified.

[0013] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement a data classification method as described in the first aspect and any possible implementation thereof.

[0014] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the data classification method of the first aspect.

[0015] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions that, when executed on an electronic device, cause the electronic device to perform the data classification method described in the first aspect and any possible implementation thereof.

[0016] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects: Based on the correlation degree between multiple target positive historical data and other target positive historical data, the distribution interval, i.e., the confidence interval, of the target positive historical data with a high degree of correlation is determined. Thus, the confidence interval determined based on the correlation degree between target positive historical data and other target positive historical data can accurately measure the distribution interval of non-abnormal data. Furthermore, using this confidence interval as the boundary for data classification, abnormal data and / or non-abnormal data in the data to be classified are divided. Based on this, the data classification of the data to be classified is determined by using the data distribution boundary of a class of positive historical data, which can be achieved using only one class of data, simplifying the data construction process. Moreover, this data classification method avoids the influence of differences in the amount of data of different data types, improving the accuracy of data classification.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0019] Figure 1 This is a schematic diagram illustrating a data classification system according to an exemplary embodiment;

[0020] Figure 2 This is a flowchart illustrating a data classification method according to an exemplary embodiment. Figure 1 ;

[0021] Figure 3 This is a flowchart illustrating a data classification method according to an exemplary embodiment. Figure 2 ;

[0022] Figure 4 This is a block diagram illustrating a data classification apparatus according to an exemplary embodiment;

[0023] Figure 5 This is a schematic diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0026] Before providing a detailed description of the data classification method provided in the embodiments of this application, let's first briefly introduce the application scenarios and implementation architecture involved in the embodiments of this application.

[0027] First, a brief introduction to the application scenarios involved in this application will be given.

[0028] With the development of computer technology, in order to facilitate the management and application of data, it has become a trend to use machine learning models to classify different types of data (such as data of different customers in the insurance industry).

[0029] In related technologies, historical positive and negative sample data are typically used to train a data classification model. This trained model is then used to classify the input data to determine its data type. However, in this method, if the distribution of positive and negative sample data in the historical data is uneven, the classification results may exhibit classification bias. That is, the classification results may favor the data type with the larger sample size, leading to lower accuracy.

[0030] To address the aforementioned issues, this application provides a data classification method that determines the distribution interval, or confidence interval, of the highly correlated positive historical data of multiple targets based on the degree of correlation with other positive historical data of the targets. Thus, the confidence interval determined based on the correlation between the positive historical data of the targets and other positive historical data of the targets can accurately measure the distribution interval of non-anomaly data. Furthermore, this confidence interval is used as the boundary for data classification to separate abnormal and / or non-abnormal data within the data to be classified.

[0031] Based on the above data classification method, the data classification of the data to be classified is determined by obtaining the data distribution boundary of a class of positive historical data. This can be achieved using only one class of data, simplifying the data construction process. Furthermore, this data classification method avoids the influence of differences in the amount of data of different data types, thus improving the accuracy of data classification.

[0032] Secondly, the implementation architecture involved in this application will be briefly introduced below.

[0033] Figure 1 This is a schematic diagram of a data classification system 10 provided in this disclosure. For example... Figure 1 As shown, the data classification system includes a server 101 and a terminal device 102, which can establish a connection through a wired network or a wireless network.

[0034] In some embodiments, server 101 uses positive historical data from different users as input and confidence intervals corresponding to each positive historical data as output to construct a preset model, so that the preset model has the function of determining confidence intervals for multiple positive historical data.

[0035] Furthermore, when a user's set of data to be classified is input into terminal device 102, terminal device 102 sends a classification request to server 101 to classify the data to be classified. Server 102 accepts the classification request, determines multiple positive historical data sets for multiple users consistent with the user type, and inputs the determined multiple positive historical data sets into a preset model to obtain a confidence interval consistent with the data type of the data to be classified. At the same time, server 102 classifies the data to be classified using the confidence interval and sends the classification result to terminal device 102.

[0036] In some embodiments, server 101 includes or is connected to a database, and positive historical data of different user types or different data types can be stored in the database. Terminal devices can access the positive historical data in the database through server 101.

[0037] In other embodiments, server 101 may be a single server, or it may be a server cluster consisting of multiple servers. In some embodiments, the server cluster may also be a distributed cluster. This application does not limit the specific implementation of server 101.

[0038] The terminal device can be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc., that can install and use content community applications (such as Kuaishou). This disclosure does not impose any special restrictions on the specific form of the terminal device. It can interact with users through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction, or handwriting device.

[0039] Optionally, the above Figure 1 In the data classification system shown, server 101 can be connected to at least one terminal device. This application does not limit the number or type of terminal devices.

[0040] The data classification method provided in this application embodiment can be applied to the aforementioned... Figure 1 The server in the data classification system shown in the implementation architecture is illustrated. For ease of understanding, the data classification method provided in this application will be described in detail below with reference to the accompanying drawings.

[0041] Figure 2 This is a flowchart illustrating a data classification method according to an exemplary embodiment, such as... Figure 2 As shown, this data classification method includes the following steps.

[0042] S21, the server determines the confidence interval based on the confidence levels of multiple positive historical data of the target.

[0043] Among them, the target positive historical data can be positive historical data directly obtained from multiple positive historical data in the historical data sample set, or it can be positive historical data obtained by processing the positive historical data.

[0044] All the aforementioned positive historical data for the targets are non-anomaly data. Non-anomaly data refers to data that conforms to a positive development trend. A positive development trend can also be described as a "data distribution trend without anomalies." It can be understood that any target's positive historical data is derived from multiple data points that constitute a positive development trend.

[0045] The confidence level described above represents the degree of correlation between the positional distribution of each target's positive historical data and other target's positive historical data across multiple positive historical data sets. The confidence interval represents the distribution range of target's positive historical data with a confidence level higher than a preset confidence level.

[0046] This preset confidence level is the threshold used to distinguish between high and low confidence levels.

[0047] Specifically, the distribution interval of positive historical data belonging to multiple targets with a correlation higher than a certain level is defined as the confidence interval. The higher the correlation (i.e., the higher the confidence level) of the positive historical data of a target, the denser the data distribution of that target's positive historical data, and the better that target's positive historical data can measure the data to be classified, thus improving the accuracy of the confidence interval.

[0048] In this step, multiple target positive historical data sets are consistent with the data to be classified in terms of data type, corresponding business type, or user type. Based on the correspondence between multiple target positive historical data sets and the data to be classified, multiple positive historical data sets corresponding to the data to be classified are determined to ensure consistency between the multiple target positive historical data sets and the data to be classified. This ensures that the confidence interval determined based on the multiple target positive historical data sets can measure the location distribution of the data to be classified, thereby ensuring the accuracy of data classification.

[0049] S22, the server classifies the data to be classified according to the confidence interval to determine the abnormal data and / or non-abnormal data in the data to be classified.

[0050] Specifically, if any data in the data to be classified does not fall within the confidence interval, it indicates that the data in the data to be classified is outlier. If any data in the data to be classified falls within the confidence interval, it indicates that the outlier data in the data to be classified is non-outlier. If all data in the data to be classified falls within the confidence interval, it indicates that there is no outlier data in the data to be classified, and all data are non-outlier.

[0051] In this step, the confidence interval is used as the boundary for classifying data as non-abnormal data, and the abnormal data and / or non-abnormal data in the data to be classified are divided.

[0052] In some embodiments, when it is determined that the data to be classified includes abnormal data, an early warning message is issued to alert the user of the terminal device that there is abnormal data, so that the user can take corresponding early warning measures based on the abnormal data.

[0053] For example, taking the data to be classified as the target insurance data, the data classification process will be further explained.

[0054] The insurance data must include at least one of the following: customer needs information, business information, customer credit information, and historical claims information. Specifically, customer needs information includes: insurance needs, whether the customer is a key client, the policyholder's consecutive renewal years, overall profitability, registered capital, and number of employees. Business information includes: product lines, total premiums, initial premium payment ratio, policy term, start date, and end date. Customer credit information primarily uses external credit data to rate the creditworthiness of the corporate client. Historical claims information includes: frequency of claims, whether the customer has a large annual payout, and the number of claims for the corporate's products.

[0055] The server retrieves a large amount of insurance application data from other insured users. Based on this large amount of data, a confidence interval is determined. Then, based on the confidence interval, it is determined whether there is any abnormal data in the target insured user's insurance application data.

[0056] If the target insurance data includes insurance applications that do not fall within the confidence interval, it indicates the presence of abnormal insurance applications, suggesting a high probability of premium delinquency for the insured. Conversely, if the target insurance data includes only insurance applications that fall within the confidence interval, it indicates the absence of abnormal insurance applications, suggesting a low probability of premium delinquency for the insured.

[0057] Based on this, risks affecting premiums receivable can be identified in advance, allowing for decisions on whether to issue a policy or adopt different collection methods later based on the customer's insurance data, thus facilitating reasonable control over the entry point of premiums receivable.

[0058] The above data classification implementation method, based on the correlation between the target's positive historical data and other target's positive historical data, determines a confidence interval that can accurately measure the distribution range of non-anomaly-prone data. Furthermore, this confidence interval is used as the boundary for data classification to separate anomalous and / or non-anomaly data within the data to be classified. Based on this, by using the data distribution boundary of a single type of positive historical data to determine the data classification of the data to be classified, only one type of data is needed, simplifying the data construction process. Moreover, this data classification method avoids the influence of differences in data volume between different data types, improving the accuracy of data classification.

[0059] As a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process of this embodiment, in conjunction with the above embodiments, this application provides the following data classification methods.

[0060] The aforementioned positive historical data for the target can be determined from multiple positive historical data in the historical data sample set.

[0061] As a method for determining positive historical data of a target, the server uses non-negative matrix factorization to perform dimensionality reduction on multiple positive historical data to obtain multiple positive historical data of the target.

[0062] It is understandable that data features other than the non-preset dimension features included in each historical data set are deleted, while the preset dimension features of each historical data set are retained.

[0063] Based on this, a nonnegative matrix factorization algorithm is used to remove positive historical data with low correlation, thereby reducing the dimension corresponding to the target positive historical data and improving the speed of distance calculation between data, thus improving the efficiency of data classification.

[0064] When performing the above determination method, the following steps can also be performed: delete positive historical data with correlation below the correlation threshold from multiple positive historical data sets in order to optimize the correlation of the target positive historical data and improve the accuracy of the confidence interval.

[0065] As another method for determining target positive historical data, the server normalizes multiple positive historical data sets to obtain normalized positive historical data. All of the normalized positive historical data are within a preset range (non-negative (belonging to [0,1])). Furthermore, based on the normalized positive historical data, multiple target positive historical data sets are obtained.

[0066] For example, the above normalization process can be achieved by the maximum-minimum value normalization method of the following formula (1).

[0067] X new =(X i -min(X j )) / (max(X j )-min(X j )) Formula (1)

[0068] Among them, X new X represents the normalized positive historical data (positive historical data matrix). i Let X represent any positive historical data, min(X) j ) represents the minimum value in the set of features composed of the j-th feature in a feature matrix composed of multiple positive historical data; max(X) j ) represents the maximum value in the set of features composed of the j-th feature in the feature matrix composed of multiple positive historical data.

[0069] Based on the above normalization process, the influence of outliers in multiple positive historical data can be reduced, ensuring the accuracy of the target positive historical data.

[0070] The two methods for acquiring positive historical data of the target described above can be used in combination to determine the target's positive historical data. This application does not specifically limit the determination method; it should be determined according to the specific application scenario. To avoid repetition, it will not be elaborated further.

[0071] Correspondingly, the data to be classified can also be processed using the above-mentioned non-negative matrix factorization method and / or normalization method to ensure the consistency between the data to be classified and the target positive historical data.

[0072] Furthermore, as a method for acquiring positive historical data, the server groups and fits multiple sets of historical data to obtain multiple sets of fitted curves. The historical data corresponding to the fitted curves that conform to a positive development trend are identified as multiple positive historical data in the aforementioned historical data sample set.

[0073] For example, if the rate of change of the slope of the fitted curve remains within a preset range, it indicates that the fitted curve conforms to a positive development trend.

[0074] As another way to obtain positive historical data, when collecting historical data sample sets, the data of multiple high-quality users are identified as multiple positive historical data.

[0075] For example, the premium user can be a customer with a good long-term payment record selected from customers paying premiums under long-term payment services (e.g., long-term fixed-amount insurance customers). The premium user can be a user who meets one or more of the following evaluation indicators: a credit score higher than a preset credit score threshold, a credit score growth rate greater than a preset growth rate, and a user account balance greater than a preset amount.

[0076] The two methods for acquiring positive historical data described above can be used in combination to obtain positive historical data. This application does not specifically limit the acquisition method; it should be determined according to the specific application scenario. To avoid repetition, it will not be elaborated further.

[0077] As a confidence level determination method, the associated data corresponding to each target's positive historical data is identified from multiple positive historical data sets. Based on the association results between each target's positive historical data and its corresponding associated data, the confidence level of each target's positive historical data is determined.

[0078] Based on this, combined Figure 2 like Figure 3 As shown, step S21 can be implemented in the following steps.

[0079] S211, the server determines the associated data corresponding to each target's positive historical data from multiple target positive historical data.

[0080] Among them, the associated data are the first preset number of other target positive historical data that have the highest degree of correlation with the target positive historical data.

[0081] In some embodiments, the data with the closest correlation can be the data whose location distribution distance difference from the target's positive historical data is the smallest of a first preset number.

[0082] Specifically, other target positive historical data are sorted in ascending order according to the distance difference with the target positive historical data. From the other target positive historical data, other target positive historical data that are located before the first position to the first preset number of positions are selected as the associated data of the target positive historical data.

[0083] For example, taking 1000 positive historical data of a target and a first preset number of 7, the associated data of a positive historical data of a target is the 7 other positive historical data of a target that are most closely related to the positive historical data of that target among 999 other positive historical data of a target.

[0084] S212, the server determines the confidence level of each target's positive historical data by averaging the correlation between each target's positive historical data and its corresponding related data.

[0085] In some embodiments, the average result of the locational distribution distance difference between each target's positive historical data and its corresponding associated data is the average result of the degree of association between each target's positive historical data and its corresponding associated data.

[0086] For example, the average result of the correlation between the positive historical data of each target and the corresponding associated data is represented by the following formula (2).

[0087]

[0088] Where k represents the first preset quantity, This represents the positive historical data of any one of the n targets. Represents X i Positive historical data of the k other targets with the smallest distance difference from their location distribution.

[0089] This step achieves the measurement of the positional distribution trend of the target's positive historical data using a first preset number of other target positive historical data that are closest to the target's positive historical data.

[0090] S213, the server determines the confidence interval based on the confidence level of the positive historical data of each target.

[0091] In some implementations, a bootstrap method is used to determine the confidence level. Specifically, the confidence level can be set to 95%.

[0092] Optionally, in order to ensure that the calculated confidence interval is more accurate, S213 can be implemented in the following ways.

[0093] The server determines the confidence interval by averaging the target confidence scores extracted from the confidence scores of each target's positive historical data based on a preset number of extractions.

[0094] Specifically, the number of confidence scores extracted from the positive historical data of each target in each preset extraction count is the second preset number. That is, the number of target confidence scores is the second preset number.

[0095] Understandably, each time, a second preset number of target confidence levels are extracted from multiple confidence levels of positive historical data for multiple targets, and the second preset number of target confidence levels are averaged to obtain an average result. After extraction based on a preset number of times, a preset number of average results are obtained, and the confidence interval is determined based on these preset number of average results.

[0096] Based on this, multiple confidence levels corresponding to positive historical data of multiple targets are re-extracted, and the average result is taken again after extraction to ensure that the data difference is smaller, thereby ensuring that the determined confidence interval is more accurate.

[0097] As a classification method, step S22 above can be implemented in the following way: the server classifies the data to be classified based on whether the confidence level of the data to be classified belongs to the confidence interval, so as to determine the abnormal data and / or non-abnormal data in the data to be classified.

[0098] Specifically, the server first determines the candidate confidence level of each candidate data point in the data to be classified. The candidate confidence level characterizes the degree of correlation between each candidate data point and other candidate data points in the distribution of their positions within the data to be classified. Then, based on the confidence intervals, the server classifies each candidate data point according to its candidate confidence level, obtaining a classification result. Based on this classification result, outlier and / or non-outlier data points in the data to be classified are identified.

[0099] Furthermore, when the classification result indicates that the candidate confidence level of the candidate data falls within the confidence interval, the candidate data corresponding to the candidate confidence interval that falls within the confidence interval is identified as non-abnormal data. When the classification result indicates that the candidate confidence level of the candidate data is outside the confidence interval, the candidate data corresponding to the candidate confidence interval that is outside the confidence interval is identified as anomalous data.

[0100] The method for determining the candidate confidence level can refer to the confidence level determination method in the above embodiments. To avoid repetition, it will not be repeated here.

[0101] Based on this, the classification result of each data point is determined according to the relationship between the confidence level and confidence interval of each data point in the data to be classified. The processing logic is simple and the accuracy is high.

[0102] To achieve the above functions, the data classification device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art will readily recognize that, based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0103] This disclosure also provides an embodiment such as Figure 4 The data classification device shown includes a determining unit 401 and a classification unit 402.

[0104] The determining unit 401 is configured to determine a confidence interval based on the confidence level of multiple target positive historical data. The confidence level represents the degree of correlation between the positional distribution of each target positive historical data point and other target positive historical data points within the multiple target positive historical data sets; the confidence interval represents the distribution range of target positive historical data points within the multiple target positive historical data sets whose confidence level is higher than a preset confidence level.

[0105] Classification unit 402 is configured to perform classification of the data to be classified based on confidence intervals to identify outlier and / or non-outlier data in the data to be classified.

[0106] In one possible implementation, the determining unit 401 is specifically configured to perform: determining the associated data corresponding to each target positive historical data from multiple target positive historical data; the associated data being the data with the smallest positional distribution distance difference between the target positive historical data and the first preset number of other target positive historical data; determining the average result of the positional distribution distance difference between each target positive historical data and the corresponding associated data as the confidence level of each target positive historical data; and determining a confidence interval based on the confidence level of each target positive historical data.

[0107] In another possible implementation, the determining unit 401 is further specifically configured to perform: determining a confidence interval based on the average result of the target confidence scores extracted from the confidence scores of each target positive historical data based on a preset number of extractions; the number of confidence scores extracted from the confidence scores of each target positive historical data each time in the preset number of extractions is a second preset number.

[0108] In another possible implementation, the classification unit 402 is specifically configured to perform: determining the candidate confidence level of each candidate data in the data to be classified; the candidate confidence level characterizes the degree of correlation between each candidate data and other candidate data in the location distribution of the data to be classified; classifying the candidate confidence level of each candidate data according to the confidence interval to obtain the classification result; and determining the abnormal data and / or non-abnormal data in the data to be classified according to the classification result.

[0109] In another possible implementation, the classification unit 402 is further specifically configured to perform the following: when the classification result indicates that the candidate confidence level of the candidate data belongs to the confidence interval, the candidate data corresponding to the candidate confidence interval that belongs to the confidence interval is determined as non-abnormal data; when the classification result indicates that the candidate confidence level of the candidate data is outside the confidence interval, the candidate data corresponding to the candidate confidence interval outside the confidence interval is determined as abnormal data.

[0110] In another possible implementation, the determining unit 401 is further specifically configured to perform: obtaining multiple positive historical data from a set of historical data samples; and performing dimensionality reduction processing on the multiple positive historical data based on non-negative matrix factorization to obtain multiple target positive historical data.

[0111] In another possible implementation, the determining unit 401 is further specifically configured to perform: obtaining multiple positive historical data from a historical data sample set; normalizing the multiple positive historical data to obtain normalized historical data, wherein the normalized historical data are all within a preset range; and obtaining multiple target positive historical data based on the normalized historical data.

[0112] Regarding the apparatus in the above embodiments, the specific manner in which each unit module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0113] Figure 5 This is a schematic diagram of an electronic device provided in this application. (For example...) Figure 5 The electronic device 50 may include at least one processor 501 and a memory 503 for storing processor-executable instructions. The processor 501 is configured to execute the instructions in the memory 503 to implement the data classification method described in the following embodiments.

[0114] In addition, electronic device 50 may also include communication bus 502, at least one communication interface 504, input device 506 and output device 505.

[0115] Processor 501 may be a processor (central processing unit, CPU), microprocessor unit, ASIC, or one or more integrated circuits for controlling the execution of programs according to the present application.

[0116] The communication bus 502 may include a path for transmitting information between the aforementioned components.

[0117] Communication interface 504 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0118] Input device 506 is used to receive input signals and output device 505 is used to output signals.

[0119] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0120] The memory 503 stores instructions for executing the scheme of this application, and the processor 501 controls the execution. The processor 501 executes the instructions stored in the memory 503 to implement the functions of the method of this application.

[0121] In a specific implementation, as one example, the processor 501 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 in the CPU.

[0122] In a specific implementation, as one example, the electronic device 50 may include multiple processors, such as... Figure 5Processors 501 and 507 are shown in the diagram. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0123] The electronic device is as follows Figure 5 The diagram includes a processor 501 and a memory 503 for storing executable instructions of the processor 501; wherein the processor 501 is configured to execute executable instructions to implement the data classification method as described in any of the possible embodiments above. And it can achieve the same technical effect, so to avoid repetition, it will not be described again here.

[0124] This application also provides a computer-readable storage medium, which, when executed by a processor of a data classification device or electronic device, enables the data classification device or electronic device to perform a data classification method as described in any of the possible embodiments above. The same technical effects can be achieved, and to avoid repetition, further details are omitted here.

[0125] This application also provides a computer program product, including a computer program or instructions, which are executed by a processor as a data classification method according to any of the possible implementations described above. Furthermore, it achieves the same technical effects, and to avoid repetition, will not be described again here.

[0126] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0127] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data classification method, characterized in that, The method includes: A confidence interval is determined based on the confidence levels of multiple target positive historical data; the confidence level represents the degree of correlation between the positional distribution of each target positive historical data and other target positive historical data in the multiple target positive historical data; the confidence interval represents the distribution range of target positive historical data with a confidence level higher than a preset confidence level. The data to be classified is classified according to the confidence interval to determine the abnormal data and / or non-abnormal data in the data to be classified. The process of determining the confidence interval based on the confidence levels of multiple positive historical data of targets includes: From the multiple target positive historical data, determine the associated data corresponding to each target positive historical data; the associated data are the data with the smallest positional distribution distance difference between the target positive historical data and the other target positive historical data, which are a first preset number of such data. The average of the locational distribution distance differences between the positive historical data of each target and the corresponding associated data is determined as the confidence level of the positive historical data of each target. The confidence interval is determined based on the confidence level of the positive historical data of each target; Determining the confidence interval based on the confidence level of the positive historical data of each target includes: The confidence interval is determined by the average result of the target confidence scores extracted from the confidence scores of each target positive historical data based on a preset number of extractions; the number of confidence scores extracted from the confidence scores of each target positive historical data in each preset number of extractions is a second preset number.

2. The method according to claim 1, characterized in that, The step of classifying the data to be classified according to the confidence interval to determine outlier and / or non-outlier data in the data to be classified includes: Determine the candidate confidence level for each candidate data in the data to be classified; the candidate confidence level characterizes the degree of correlation between each candidate data and other candidate data in the positional distribution of the data to be classified. The candidate confidence scores of each candidate data point are classified according to the confidence intervals to obtain the classification results; Based on the classification results, abnormal data and / or non-abnormal data in the data to be classified are determined.

3. The method according to claim 2, characterized in that, The step of determining abnormal and / or non-abnormal data in the data to be classified based on the classification result includes: When the classification result indicates that the candidate confidence level of the candidate data belongs to the confidence interval, the candidate data corresponding to the candidate confidence interval that belongs to the confidence interval is determined as non-abnormal data; When the classification result indicates that the candidate confidence level of the candidate data is outside the confidence interval, the candidate data corresponding to the candidate confidence interval outside the confidence interval is identified as abnormal data.

4. The method according to claim 1, characterized in that, The method further includes: Obtain multiple positive historical data sets from a historical data sample set; Based on nonnegative matrix factorization, the multiple positive historical data are dimensionality reduced to obtain the multiple target positive historical data.

5. The method according to claim 1, characterized in that, The method further includes: Obtain multiple positive historical data sets from a historical data sample set; The multiple positive historical data are normalized to obtain normalized historical data, and the normalized historical data are all within a preset range. Based on the normalized historical data, the positive historical data of the multiple targets are obtained.

6. A data classification device, characterized in that, The device, used in a consultation platform, includes: The determining unit is configured to perform the determination of a confidence interval based on the confidence level of multiple target positive historical data; the confidence level represents the degree of correlation between the positional distribution of each target positive historical data and other target positive historical data in the multiple target positive historical data; the confidence interval represents the distribution range of target positive historical data with a confidence level higher than a preset confidence level; The classification unit is configured to perform classification of the data to be classified based on the confidence interval to determine outlier and / or non-outlier data in the data to be classified. The determining unit is specifically configured as follows: From the plurality of target positive historical data, determine the associated data corresponding to each target positive historical data; the associated data are the data with the smallest positional distribution distance difference between the target positive historical data and the target positive historical data among the other target positive historical data, which is a first preset number of such data; the average result of the positional distribution distance difference between each target positive historical data and the corresponding associated data is determined as the confidence level of each target positive historical data; the confidence interval is determined based on the confidence level of each target positive historical data. The determining unit is further specifically configured as follows: The confidence interval is determined by the average result of the target confidence scores extracted from the confidence scores of each target positive historical data based on a preset number of extractions; the number of confidence scores extracted from the confidence scores of each target positive historical data in each preset number of extractions is a second preset number.

7. An electronic device, characterized in that, include: A processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the data classification method as described in any one of claims 1-5.

8. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform any one of the data classification methods as claimed in claims 1-5.

Images