Label quality intelligent analysis method and device, electronic equipment and storage medium

Abstract

The application relates to intelligent decision-making technology in the financial field, and discloses an intelligent label quality analysis method which can be used for quality evaluation of user labels of a financial product and comprises the following steps: according to a business scenario, sample data corresponding to class labels of a label system to be evaluated is extracted; an initial feature data set with a distinguishing degree in the sample data is selected, information gain between the initial feature data set and the class labels is calculated, and a target feature data set of the class labels is selected from the initial feature data set; label test data of the target feature data set is constructed, and predicted labels of the label test data are identified; a label quality analysis index of the label system is inquired, and index scores of the label quality analysis index are calculated according to the predicted labels and the label test data; according to the index scores, quality scores of each label in the label system are calculated, and a label quality analysis report of the label system is generated. The application can improve the accuracy of label quality analysis of the financial product.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent decision-making, and in particular to a label quality intelligent analysis method and device, an electronic device and a storage medium. BACKGROUND

[0002] In recent years, with the progress and rapid development of information technology, we have entered the era of big data, which has a subtle impact on our daily life and has become a valuable and rich asset. For enterprises, especially financial companies, how to use big data to better serve users requires a comprehensive understanding of users and the establishment of a complete and usable user label system to help financial companies describe and portray users from different dimensions, so as to better execute financial product recommendations, such as recommending insurance products or financial products. Label refers to the classification or content of the target. Label quality is used to measure the pros and cons of the label system in solving user needs.

[0003] Currently, the label quality evaluation in the industry is mostly measured by saturation and label usage, and there is no clear measurement index for the reasonableness of the value segmentation of some labels, the accuracy of user label, and the measurement of data fluctuations. For financial products with many customer classification dimensions, there is no comprehensive evaluation of the label quality of the user label system from multiple indexes, which affects the accuracy of label quality analysis. SUMMARY

[0004] The present application provides a label quality intelligent analysis method, device, electronic device and storage medium, which mainly aims to improve the accuracy of label quality analysis of user labels of financial products.

[0005] To achieve the above purpose, the present application provides a label quality intelligent analysis method, which comprises:

[0006] Identifying the business scenario of the label system to be evaluated and the corresponding class label, extracting the sample data corresponding to the class label according to the business scenario;

[0007] Selecting a data set with discrimination in the sample data as an initial feature data set, calculating the information gain between the initial feature data set and the class label, and selecting a target feature data set of the class label from the initial feature data set according to the information gain;

[0008] Constructing label test data of the target feature data set, and identifying the predicted label of the label test data by using the label classifier in the trained label analysis model;

[0009] query a label quality analysis index of the label system, and calculate the index score of the label quality analysis index according to the predicted label, the label test data, and the label quality analysis function in the trained label analysis model;

[0010] According to the index score, calculate the quality score of each label in the label system, and generate a label quality analysis report of the label system according to the quality score.

[0011] Optionally, the calculation of the information gain between the initial feature data set and the class label includes:

[0012] The information gain between the initial feature data set and the class label is calculated by the following formula:

[0013] IG(f i ;T) = H(f i ) + H(T) - H(f i ,T)

[0014] Where, IG(f i ;T) represents the information gain between the i-th feature f i in the initial feature data set and the class label set T, H(f i ) represents the information entropy of the i-th feature f i , H(T) represents the information entropy of the class label set T, and H(f i ,T) represents the joint information entropy of the i-th feature f i and the class label set T.

[0015] Optionally, the selection of the target feature data set of the class label from the initial feature data set according to the information gain includes:

[0016] The information gain is normalized to obtain a normalized gain.

[0017] The mean value of the normalized gain is calculated, and the initial feature data set corresponding to the feature not less than the threshold value is selected as the target feature data set.

[0018] Optionally, the normalization of the information gain to obtain the normalized gain includes:

[0019] The normalization of the information gain is as follows:

[0020]

[0021] Where, SU(f i ,T) represents the i-th feature f inormalized information gain between the i-th feature f i and the class label set T, IG(f i ) represents the information gain between the i-th feature f i and the class label set T, H(f i ) represents the information entropy of the i-th feature f t , and H(T) represents the information entropy of the class label set T.

[0022] Optionally, the constructing of the label classifier comprises:

[0023]

[0024] wherein γ t represents the weight vector of the t-th training iteration of the label classifier, t represents the number of training iterations of the label classifier, γ t-1 represents the weight vector of the (t-1)-th training iteration of the label classifier, x i represents the i-th training sample data vector, and p represents the label weight learning rate.

[0025] Optionally, the signature quality analysis function comprises:

[0026]

[0027] wherein Acc represents the accuracy score of the label prediction, R j represents the true label of the j-th label test data, Z j represents the predicted label corresponding to the j-th label test data, |R j ∩Z j | represents the number of correctly predicted labels, |R j ∪Z j | represents the total number of true labels and predicted labels, ∩ represents the intersection symbol, ∪ represents the union symbol, and n represents the number of label test data.

[0028] Optionally, the calculating of the quality score of each label in the label system according to the indicator score comprises:

[0029] uniformizing the indicator corresponding to the indicator score to obtain an indicator uniformization score, and performing data standardization processing on the indicator uniformization score to obtain an indicator standardization score;

[0030] analyzing the importance of the indicator in the business scenario of the label system, and assigning a weight coefficient to the indicator according to the importance;

[0031] According to the weight coefficient and the index standardization score, a product sum of the weight coefficient and the index score is calculated to obtain a quality score of each label in the label system.

[0032] To solve the above problems, the application further provides a label quality intelligent analysis device, which comprises:

[0033] A class label identification module is configured to identify a business scenario of a label system to be evaluated and a corresponding class label thereof, and extract sample data corresponding to the class label according to the business scenario;

[0034] A target feature data set generation module is configured to select a data set with a distinguishing degree in the sample data as an initial feature data set, calculate information gain between the initial feature data set and the class label, and select a target feature data set of the class label from the initial feature data set according to the information gain;

[0035] A predicted label identification module is configured to construct label test data of the target feature data set, and identify a predicted label of the label test data by using a label classifier in a trained label analysis model;

[0036] An index score calculation module is configured to query a label quality analysis index of the label system, calculate an index score of the label quality analysis index by using a label quality analysis function in the trained label analysis model according to the predicted label and the label test data;

[0037] A label quality analysis report generation module is configured to calculate a quality score of each label in the label system according to the index score, and generate a label quality analysis report of the label system according to the quality score.

[0038] To solve the above problems, the application further provides an electronic device, which comprises:

[0039] at least one processor; and

[0040] a memory in communication connection with the at least one processor; wherein

[0041] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to implement the label quality intelligent analysis method described above.

[0042] To solve the above problems, the application further provides a computer readable storage medium, which stores at least one computer program, and the at least one computer program is executed by a processor in an electronic device to implement the label quality intelligent analysis method described above.

[0043] The embodiment of the present application provides the operation object for subsequent label quality evaluation by class label identification on the label in the label system to be evaluated, and extracts the sample data corresponding to the class label to preliminarily remove the redundant data in the original data, and selects the data set with the distinguishing degree in the sample data as the initial feature data set to further effectively remove the redundant data in the sample data and retain the feature data set with higher classification ability. Secondly, the embodiment of the present application can measure the association degree of each feature in the initial feature data set and the label set by calculating the information gain between the initial feature data set and the class label, and select the target feature data set of the class label from the initial feature data set according to the information gain to more effectively improve the classification performance of the subsequent multi-label classifier. The label test data of the target feature data set is constructed to provide the operation object for the classification effect of the subsequent test label classifier, and the prediction label of the label test data is recognized by using the label classifier in the trained label analysis model to obtain the label test result to measure the classification quality of the label classifier. Further, the embodiment of the present application can help to control the label quality by querying the label quality analysis index of the label system, guide the manager and developer of the label to continuously improve the label quality to better serve the user, calculate the index score of the label quality analysis index by using the label quality analysis function in the trained label analysis model according to the prediction label and the label test data, and establish a direct quantitative analysis of the label quality of the label system according to the index score. According to the index score, the quality score of each label in the label system is calculated to judge the practical value of the label system, to provide protection for further dynamic adjustment of the label of the label system, and to generate the label quality analysis report of the label system according to the quality score to more directly and quantitatively analyze the label quality, to improve the accuracy of label classification of the label system and to more accurately serve the user. Therefore, the label quality intelligent analysis method, device, electronic equipment and storage medium provided by the embodiment of the present application can improve the customer label quality analysis accuracy of the financial product with more customer classification dimensions. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 The flowchart of the label quality intelligent analysis method provided by an embodiment of the present application is shown in the figure;

[0045] Figure 2 The module schematic diagram of the label quality intelligent analysis device provided by an embodiment of the present application is shown in the figure;

[0046] Figure 3 The internal structure schematic diagram of the electronic equipment for implementing the label quality intelligent analysis method provided by an embodiment of the present application is shown in the figure;

[0047] The objectives, functional characteristics and advantages of the present application will be further illustrated in conjunction with the embodiments with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein merely exemplify the present application and are not intended to limit the present application.

[0049] Embodiments of the present application provide a label quality intelligent analysis method. The execution subject of the label quality intelligent analysis method includes but is not limited to at least one of electronic devices such as a server, a terminal, etc., which can be configured to execute the method provided by the embodiments of the present application. In other words, the label quality intelligent analysis method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server, or a cloud server cluster, etc. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and big data and artificial intelligence platforms, etc. basic cloud computing services.

[0050] Reference Figure 1 As shown in the figure, it is a flowchart of the label quality intelligent analysis method provided by an embodiment of the present application. In the embodiment of the present application, the label quality intelligent analysis method includes:

[0051] S1, identifying the business scenario of the label system to be evaluated and the corresponding class label, extracting the sample data corresponding to the class label according to the business scenario;

[0052] In the embodiments of the present application, the application scenario of the label and the label category can be obtained by identifying the business scenario of the label system to be evaluated and the corresponding class label, which provides an operation object for subsequent evaluation of the label quality. The label system to be evaluated refers to a system that describes and characterizes users from different dimensions to achieve business requirements, such as a financial product system that can include labels describing and characterizing users from dimensions such as age, income, consumption preference, and financial habit. The label system to be evaluated can be obtained through a data script, which can be compiled through a JS script language. The label refers to a classification or content of a target sign. The business scenario is used to describe the application environment of the product or service that the user may need and is associated, such as investment and financial management, online shopping commodity push, etc. The class label is a label representing the category of data meaning, such as age, income, consumption preference, and financial habit.

[0053] Further, in an optional embodiment of the present application, the business scenario of the label system to be evaluated and the corresponding class label can be identified by analyzing the business requirements and semantic meanings of the data representation of the label system.

[0054] Further, according to the business scenario, the sample data corresponding to the class label can be extracted in the embodiment of the present application to preliminarily remove the redundant data in the original data, thereby providing support for subsequent extraction of the feature data set.

[0055] Further, according to the business scenario, the sample data corresponding to the class label can be extracted in an optional embodiment of the present application, including: analyzing the semantic meaning of the class label, filtering the redundant data in the original data of the label system according to the semantic meaning, and obtaining the sample data.

[0056] S2, selecting a data set with a distinguishing degree in the sample data as an initial feature data set, calculating the information gain between the initial feature data set and the class label, and selecting a target feature data set of the class label from the initial feature data set according to the information gain;

[0057] In the embodiment of the present application, the data set with a distinguishing degree (for example, a data set with a distinguishing degree such as user income, consumption preference, and financial habit can be selected from the sample data) in the sample data can be selected as the initial feature data set, so that the redundant data in the sample data can be effectively removed, and the feature data set with a higher classification ability can be retained, thereby improving the subsequent classification effect of the multi-label. The distinguishing degree refers to the distinguishing ability of effectively measuring or measuring certain characteristics or states to be understood. For example, the data of user income can measure the purchasing power of the user for the financial product, and therefore can be used as a data set with a distinguishing degree. The data set refers to a collection of data with a certain structure, which can be understood as containing many members, and each member has many characteristics.

[0058] Further, in an optional embodiment of the present application, the selection of the data set with a distinguishing degree in the sample data as the initial feature data set can be realized by a distinguishing degree analysis method, such as discriminant index method and correlation coefficient method.

[0059] Further, by calculating the information gain between the initial feature data set and the class label, the embodiment of the present application can measure the correlation degree of each feature in the initial feature data set and the label set, thereby providing a basis for subsequent further feature selection.

[0060] Further, in an optional embodiment of the present application, the information gain between the initial feature data set and the class label can be calculated by the following formula:

[0061] IG(fi ;T)=H(f i )+H(T)-H(f i ,T)

[0062] Among them, IG(f i ;T) represents the i-th feature f in the initial feature dataset. i (For example, the i-th feature f) i It could be the information gain H(f) between "high consumption" and a set of class labels T (e.g., a set of class labels such as age, annual income, consumption preferences, financial habits, etc.). i ) represents the i-th feature f i Information entropy, H(T) represents the information entropy of the class label set T, H(f i (T) represents the i-th feature f i The joint information entropy with the class label set T.

[0063] Furthermore, in this embodiment of the invention, selecting the target feature dataset of the class label from the initial feature dataset based on the information gain can more effectively improve the classification performance of the subsequent multi-label classifier.

[0064] Further, in an optional embodiment of the present invention, the step of selecting the target feature dataset of the class label from the initial feature dataset based on the information gain includes: normalizing the information gain to obtain a normalized gain; calculating the mean of the normalized gain; using the mean as a threshold, selecting features in the initial feature dataset that are not less than the threshold to generate the target feature dataset. For example, in an embodiment of the present invention, after calculating the normalized gains SU(f1;T), SU(f2;T), SU(f3;T), and SU(f4;T) of various datasets such as age, annual income, consumption preference, and financial habits, the mean SU(f1;T), SU(f2;T), SU(f3;T), and SU(f4;T) is calculated as SU(f2;T), SU(f3;T), and SU(f4;T). 均 The normalized gain SU(f2;T) of the dataset representing the income of that year is greater than the mean SU(f). 均 When ;T), the annual income dataset is used as the target feature dataset.

[0065] Furthermore, in an optional embodiment of the present invention, the normalization processing of the information gain can be achieved by the following formula:

[0066]

[0067] Among them, SU(f i (T) represents the i-th feature f in the initial feature dataset. i The normalized information gain between the class label set T and the class label set T, IG(fi T) represents the i-th feature f i information gain between the feature f i ) represents the information entropy of the i-th feature f i H(T) represents the information entropy of the class label set T.

[0068] S3, constructing label test data of the target feature data set, and identifying the predicted label of the label test data by using the label classifier in the trained label analysis model.

[0069] In the embodiment of the present application, the label test data of the target feature data set is constructed to provide an operation object for testing the classification effect of the label classifier. The label test data refers to data used to test the label classification effect and quality of the label classifier in the neural network model.

[0070] Further, in an optional embodiment of the present application, the label test data of the target feature data set can be constructed by screening data with the semantic features as the label test data according to the semantic features of the target feature data set.

[0071] Further, in the embodiment of the present application, the predicted label of the label test data can be obtained by identifying the predicted label of the label test data by using the label classifier in the trained label analysis model, so as to measure the classification quality of the label classifier. The label analysis model is a highly parallel system model with strong adaptive learning ability, which does not depend on the mathematical model of the research object, has good robustness to the system parameter changes of the controlled object and external interference, and includes an input layer, a hidden layer and an output layer. The different layers are fully connected. The label classifier is used to judge the class to which a new observation sample belongs on the basis of the training data with labeled class categories, such as decision tree classifier, selection tree classifier and naive Bayes classifier.

[0072] Further, in an optional embodiment of the present application, the predicted label of the label test data can be obtained by identifying the predicted label of the label test data by using the label classifier in the trained label analysis model, so as to measure the classification quality of the label classifier. The label analysis model is a highly parallel system model with strong adaptive learning ability, which does not depend on the mathematical model of the research object, has good robustness to the system parameter changes of the controlled object and external interference, and includes an input layer, a hidden layer and an output layer. The different layers are fully connected. The label classifier is used to judge the class to which a new observation sample belongs on the basis of the training data with labeled class categories, such as decision tree classifier, selection tree classifier and naive Bayes classifier.

[0073] Further, in an optional embodiment of the present application, the label classifier is constructed, comprising:

[0074]

[0075] wherein γ t represents the t-th training iteration weight vector of the label classifier, t represents the training iteration number of the label classifier, and γ t-1x represents a weight vector of the t-1th training iteration of the label classifier i represents the i-th training sample data vector, represents a label weight learning rate, and T represents a transposition operator of a vector.

[0076] S4, querying a label quality analysis index of the label system, calculating an index score of the label quality analysis index by using a label quality analysis function in the trained label analysis model according to the predicted label and the label test data.

[0077] In the embodiment of the present application, querying the label quality analysis index of the label system helps to control the label quality and guide the label manager and developer to continuously improve the label quality to better serve the user. The label quality analysis index refers to a standard that can measure the pros and cons of the label system, such as concentration, stability, accuracy, richness, saturation, application personnel coverage, usage, and label business value.

[0078] Further, the querying of the label quality analysis index of the label system in an optional embodiment of the present application can be realized by analyzing the business scenarios and business demands of the label system to determine the associated label quality analysis index.

[0079] Further, in the embodiment of the present application, the index score of the label quality analysis index can be calculated by using the label quality analysis function in the trained label analysis model according to the predicted label and the label test data, which can establish a direct quantitative analysis of the label quality of the label system to provide protection for the label adjustment decision of the label system developer. The label quality analysis function is an index function used to represent the label quality to evaluate the classification performance of the label in the label system, such as the accuracy function, the Hamming loss function, the precision function, and the recall function.

[0080] Further, according to the predicted label and the label test data, the index score of the label quality analysis index can be calculated by using the label quality analysis function in the trained label analysis model in an optional embodiment of the present application, and the label quality analysis function is as follows:

[0081]

[0082] wherein, Acc represents the accuracy score of the label prediction, R j represents the true label of the j-th label test data, and Z j represents the predicted label corresponding to the j-th label test data, |R j ∩Z j | represents the number of correctly predicted labels, |R j ∪Z j| represents the total number of true labels and predicted labels, represents the intersection symbol, represents the union symbol, and n represents the number of label test data.

[0083] S5. Calculate a quality score of each label in the label system according to the index score, and generate a label quality analysis report of the label system according to the quality score.

[0084] In the embodiment of the present application, the quality score of each label in the label system is calculated according to the index score, and the quality score of the entire label system is obtained to evaluate the practical value of the label system and provide a guarantee for further dynamic adjustment of the labels in the label system.

[0085] Further, the calculation of the quality score of each label in the label system according to the index score in an optional embodiment of the present application comprises: uniformization processing of the index corresponding to the index score to obtain an index uniformization score, data standardization processing of the index uniformization score to obtain an index standardization score; analysis of the importance of the index in the business scenario of the label system, and allocation of a weight coefficient of the index according to the importance; calculation of the product sum of the weight coefficient and the index score according to the weight coefficient and the index standardization score to obtain the quality score of each label in the label system. The uniformization processing of the index refers to the process of converting indexes of different properties (including positive indexes and inverse indexes) into indexes of the same property, and the uniformization processing of the inverse index can be realized by inverse uniformization, subtraction uniformization, etc. The standardization processing refers to the conversion of original data according to a certain proportion through a certain mathematical transformation method, so that it falls within a specific interval, such as [0, 1], [-1, 1], etc. The weight coefficient can be set according to the business scenario, or generated by a random function.

[0086] Further, the data standardization processing in an optional embodiment of the present application can be realized by data standardization methods such as min-max standardization, log function conversion, atan function conversion, z-score standardization, and fuzzy quantization.

[0087] Further, in the embodiment of the present application, the generation of the label quality analysis report of the label system according to the quality score can more directly and quantitatively analyze the label quality, so as to improve the accuracy of label classification of the label system and provide more accurate services for users.

[0088] Further, according to the quality score, an optional embodiment of the present application generates a label quality analysis report of the label system, comprising: constructing a relationship pair matrix of the index score and the corresponding weight, and marking the class label of the label system corresponding to the index score; and generating a label quality analysis report of the label system according to the relationship pair matrix, the class label and the quality score.

[0089] It can be seen that, by identifying the class label of the label system to be evaluated, the embodiment of the present application provides an operation object for subsequent label quality evaluation, and by extracting sample data corresponding to the class label according to the business scenario, the embodiment of the present application can preliminarily remove redundant data in the original data, and select a data set with a distinguishing degree in the sample data as an initial feature data set to further effectively remove redundant data in the sample data and retain a feature data set with higher classification ability; secondly, by calculating the information gain between the initial feature data set and the class label, the embodiment of the present application can measure the degree of association between each feature in the initial feature data set and the label set, and according to the information gain, the target feature data set of the class label is selected from the initial feature data set to more effectively improve the classification performance of the subsequent multi-label classifier, by constructing label test data of the target feature data set to provide an operation object for subsequent test of the classification effect of the label classifier, and by using the label classifier in the trained label analysis model to identify the predicted label of the label test data, a label test result can be obtained to measure the classification quality of the label classifier. Further, by querying the label quality analysis indicators of the label system, the embodiment of the present application helps to control label quality and guide label managers and developers to continuously improve label quality to better serve users, by using the label quality analysis function in the trained label analysis model to calculate the index score of the label quality analysis indicators according to the predicted label and the label test data, an intuitive and quantitative analysis of the label quality of the label system can be established, by calculating the quality score of each label in the label system according to the index score to judge the practical value of the label system, and by generating a label quality analysis report of the label system according to the quality score, the label quality can be more intuitively and quantitatively analyzed to improve the accuracy of label classification of the label system and more accurately serve users. Therefore, the label quality intelligent analysis method, device, electronic equipment and storage medium provided by the embodiment of the present application can improve the customer label quality analysis accuracy of a financial product with more customer classification dimensions.

[0090] As shown in Figure 2 , it is a functional module diagram of the label quality intelligent analysis device of the present application.

[0091] The label quality intelligent analysis device 100 can be installed in an electronic device. According to the functions implemented, the label quality intelligent analysis device can include a class label identification module 101, a target feature data set generation module 102, a predicted label identification module 103, an index score calculation module 104, and a label quality analysis report generation module 105. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by the processor of an electronic device and can complete a fixed function, which are stored in the memory of the electronic device.

[0092] In the present embodiment, the functions of each module / unit are as follows:

[0093] The class label identification module 101 is configured to identify the business scenarios of the label system to be evaluated and the corresponding class labels thereof, and extract sample data corresponding to the class labels according to the business scenarios;

[0094] The target feature data set generation module 102 is configured to select a data set with a degree of discrimination in the sample data as an initial feature data set, calculate the information gain between the initial feature data set and the class labels, and select a target feature data set of the class labels from the initial feature data set according to the information gain;

[0095] The predicted label identification module 103 is configured to construct label test data of the target feature data set, and identify a predicted label of the label test data using a label classifier in a trained label analysis model;

[0096] The index score calculation module 104 is configured to query label quality analysis indexes of the label system, calculate index scores of the label quality analysis indexes using a label quality analysis function in the trained label analysis model according to the predicted label and the label test data;

[0097] The label quality analysis report generation module 105 is configured to calculate a quality score of each label in the label system according to the index scores, and generate a label quality analysis report of the label system according to the quality scores.

[0098] In detail, the modules in the label quality intelligent analysis device 100 in the present embodiment use the same technical means as the label quality intelligent analysis method described in the above Figure 1 , and can produce the same technical effects, which will not be described here.

[0099] As shown in Figure 3 , it is a structural schematic diagram of an electronic device 1 for implementing the label quality intelligent analysis method of the present application.

[0100] The electronic device 1 can include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and can further include a computer program, such as a label quality intelligent analysis program, stored in the memory 11 and executable on the processor 10.

[0101] The processor 10 can be composed of an integrated circuit in some embodiments, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The processor 10 is the control core of the electronic device 1, and connects various components of the entire electronic device 1 through various interfaces and lines, executes or runs programs or modules stored in the memory 11 (such as executing a label quality intelligent analysis program, etc.), and calls data stored in the memory 11 to perform various functions and process data of the electronic device 1.

[0102] The memory 11 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 can be an internal storage unit of the electronic device 1 in some embodiments, for example, a mobile hard disk of the electronic device 1. The memory 11 can also be an external storage device of the electronic device 1 in other embodiments, for example, a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used not only to store application software and various data installed on the electronic device 1, such as the code of a label quality intelligent analysis program, but also to temporarily store data that has been output or will be output.

[0103] The communication bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0104] The communication interface 13 is used for communication between the electronic device 1 and other devices, including network interface and employee interface. Optionally, the network interface can include wired interface and / or wireless interface (such as WI-FI interface, Bluetooth interface, etc.), which is usually used to establish a communication connection between the electronic device 1 and other electronic devices 1. The employee interface can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the employee interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, etc. Among them, the display can also be appropriately called a display screen or a display unit, which is used to display the information processed in the electronic device 1 and to display the visual employee interface.

[0105] Figure 3 Only the electronic device 1 with components is shown, and those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0106] For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) for powering each component. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that functions such as charge management, discharge management, and power consumption management can be realized through the power management device. The power supply can also include one or more direct current or alternating current power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not described here.

[0107] It should be understood that the embodiments are only for illustration and do not limit the scope of the patent invention.

[0108] The label quality intelligent analysis program stored in the memory 11 in the electronic device 1 is a combination of multiple computer programs, which, when running in the processor 10, can realize:

[0109] Identify the business scenario of the label system to be evaluated and its corresponding class label, and extract the sample data corresponding to the class label according to the business scenario;

[0110] select a data set with discriminativeness in the sample data as an initial feature data set, calculate information gain between the initial feature data set and the class label, and select a target feature data set of the class label from the initial feature data set according to the information gain;

[0111] construct label test data of the target feature data set, and recognize a predicted label of the label test data by using a label classifier in the trained label analysis model;

[0112] query a label quality analysis index of the label system, calculate an index score of the label quality analysis index by using a label quality analysis function in the trained label analysis model according to the predicted label and the label test data;

[0113] calculate a quality score of each label in the label system according to the index score, and generate a label quality analysis report of the label system according to the quality score.

[0114] Specifically, the specific implementation method of the processor 10 to the above computer program can refer to Figure 1 The description of related steps in the corresponding embodiments will not be repeated here.

[0115] Further, the modules / units integrated in the electronic device 1, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a non-volatile computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM).

[0116] The application also provides a computer readable storage medium, which stores a computer program. When the computer program is executed by the processor of the electronic device 1, the following steps can be implemented:

[0117] identify a business scenario of a label system to be evaluated and a corresponding class label thereof, and extract sample data corresponding to the class label according to the business scenario;

[0118] select a data set with discriminativeness in the sample data as an initial feature data set, calculate information gain between the initial feature data set and the class label, and select a target feature data set of the class label from the initial feature data set according to the information gain;

[0119] constructing label test data of the target feature data set, identifying a predicted label of the label test data by using a label classifier in the trained label analysis model;

[0120] querying a label quality analysis index of the label system, calculating an index score of the label quality analysis index by using a label quality analysis function in the trained label analysis model according to the predicted label and the label test data;

[0121] calculating a quality score of each label in the label system according to the index score, and generating a label quality analysis report of the label system according to the quality score.

[0122] In several embodiments provided in the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the modules is merely a logical function division. There can be another division manner in actual implementation.

[0123] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical units. That is, they can be located in one place or distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0124] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software function module.

[0125] It is apparent for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application.

[0126] Therefore, the embodiments should be considered in all aspects as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and range of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be considered as limiting the claims to which they belong.

[0127] Embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0128] Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or devices can also be presented by a single unit or device as by means of software or hardware. The word "coupled" is used to describe either a direct or indirect connection.

[0129] Finally, it should be noted that the above-mentioned embodiments are merely intended to illustrate the technical solutions of the present application, rather than limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalent replaced without departing from the spirit and scope of the present application.

Claims

1. A method of intelligent analysis of label quality, characterized in that, The method comprises: identifying a business scenario of a label system to be evaluated and a corresponding class label thereof, and extracting sample data corresponding to the class label according to the business scenario; selecting a data set with a distinguishing degree in the sample data as an initial feature data set, calculating information gain between the initial feature data set and the class label, and selecting a target feature data set of the class label from the initial feature data set according to the information gain; constructing label test data of the target feature data set, and identifying a predicted label of the label test data by using a label classifier in a trained label analysis model; inquiring a label quality analysis index of the label system, calculating an index score of the label quality analysis index by using a label quality analysis function in the trained label analysis model according to the predicted label and the label test data; calculating a quality score of each label in the label system according to the index score, and generating a label quality analysis report of the label system according to the quality score; wherein the calculation of the information gain between the initial feature data set and the class label comprises: calculating the information gain between the initial feature data set and the class label by using the following formula: in, Represents the first feature in the initial feature dataset. Features With class tag collection Information gain between Indicates the first Features Information entropy Represents a collection of class tags Information entropy Indicates the first Features With class tag collection The joint information entropy.

2. The label quality intelligent analysis method of claim 1, wherein, the selection of the target feature data set of the class label from the initial feature data set according to the information gain comprises: normalizing the information gain to obtain a normalized gain; calculating a mean value of the normalized gain, taking the mean value as a threshold, selecting features in the initial feature data set that are not less than the threshold, and generating a target feature data set.

3. The label quality intelligent analysis method of claim 2, wherein, the normalization of the information gain to obtain the normalized gain comprises: the normalization of the information gain is as follows: in, Represents the first feature in the initial feature dataset. Features With class tag collection Normalized gain between Represents the first feature in the initial feature dataset. Features With class tag collection Information gain between Indicates the first Features Information entropy Represents a collection of class tags Information entropy.

4. The label quality intelligent analysis method of claim 1, wherein, the construction of the label classifier comprises: wherein, represents the weight vector of the label classifier at the -1th training iteration, represents the number of training iterations of the label classifier, represents the weight vector of the label classifier at the -1th training iteration, represents the -th training sample data vector, represents the -th training sample data vector, represents the label weight learning rate, represents the transpose operator of a vector.

5. The label quality intelligent analysis method of claim 1, wherein, the label quality analysis function comprises: wherein, represents the accuracy score of the label prediction, represents the true label of the th label test data, represents the predicted label corresponding to the th label test data, represents the number of labels that are predicted correctly, represents the total number of true labels and predicted labels, represents the intersection symbol, represents the union symbol, represents the number of label test data.

6. The label quality intelligent analysis method of claim 1, wherein, the calculation of the quality score of each label in the label system according to the index score comprises: performing a normalization process on the index corresponding to the index score to obtain an index normalized score, performing a data standardization process on the index normalized score to obtain an index standardized score; analyzing an importance degree of the index in the business scenario of the label system, and assigning a weight coefficient of the index according to the importance degree; calculating a product sum of the weight coefficient and the index score according to the weight coefficient and the index standardized score to obtain the quality score of each label in the label system.

7. A label quality intelligent analysis apparatus characterized by comprising: The device comprises: a class label identification module configured to identify a business scenario of a label system to be evaluated and a corresponding class label thereof, and extract sample data corresponding to the class label according to the business scenario; a target feature data set generation module configured to select a data set with a distinguishing degree in the sample data as an initial feature data set, calculate information gain between the initial feature data set and the class label, and select a target feature data set of the class label from the initial feature data set according to the information gain; The prediction label identification module is configured to construct label test data of the target feature data set, and identify a prediction label of the label test data by using a label classifier in the trained label analysis model. The index score calculation module is configured to query a label quality analysis index of the label system, calculate an index score of the label quality analysis index by using a label quality analysis function in the trained label analysis model according to the prediction label and the label test data, and calculate a quality score of each label in the label system according to the index score. The label quality analysis report generation module is configured to generate a label quality analysis report of the label system according to the quality score. The information gain between the initial feature data set and the class label is calculated by using the following formula: The electronic device comprises: in, Represents the first feature in the initial feature dataset. Features With class tag collection Information gain between Indicates the first Features Information entropy Represents a collection of class tags Information entropy Indicates the first Features With class tag collection The joint information entropy.

8. An electronic device, comprising: at least one processor; and a memory connected with the at least one processor; wherein The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the label quality intelligent analysis method according to any one of claims 1 to 6. The computer program is executed by the processor to implement the label quality intelligent analysis method according to any one of claims 1 to 6.

9. A computer readable storage medium storing a computer program, characterized in that, ​

Images