Data interface value evaluation method based on deep learning

Abstract

The invention relates to a data interface value evaluation method based on deep learning, and belongs to the technical field of computers. Compared with an existing rule-based data interface value evaluation method, the method has the advantages that in data interface value evaluation, the deep learning technology and the Logistic multi-classification regression analysis model are utilized, the problems that traditional manual rule construction is poor in timeliness, low in coverage rate and low in efficiency are solved, the disadvantage that a traditional machine learning method is low in support dimension is overcome, and the data interface value evaluation efficiency is improved. And current and future data interface value evaluation can be judged and predicted through previous data. The method can provide data interface value evaluation on the premise of ensuring correct input of interface calling data, and improves classification performance and accuracy at the same time. Therefore, the method plays an important role in data interface value evaluation.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and specifically relates to a data interface value evaluation method based on deep learning. Background Art

[0002] The data interface value evaluation method, that is, the data interface value rating, is a process of analyzing, processing and classifying the value of different data interfaces in the system.

[0003] As the degree of enterprise informatization gradually increases, the business systems, data middle platforms and other systems in the enterprise often provide a large number of data interfaces for internal and external calls. As the number of data interfaces provided by the system increases, the cost of maintaining data interfaces increases day by day, which requires the ability to automatically judge the value of data interfaces, thereby optimizing costs and tilting resources toward valuable interfaces. However, with the large increase in the number of data interfaces provided by the system, it is becoming increasingly difficult to determine the value of the interfaces, so data interface value evaluation technology is receiving more and more attention. Data interface value evaluation is an important task in the construction of data middle platforms and data services. It is mainly used for interface value maintenance, optimizing data maintenance costs, and reducing the difficulty of data operation and maintenance. It plays a vital role in the construction of enterprise data middle platforms and data resource sharing.

[0004] As data interfaces become increasingly complex and their number increases, traditional data interface value evaluation methods, namely manual rule writing methods and traditional machine learning methods, are increasingly unable to meet the growing efficiency requirements.

[0005] The traditional rule identification method uses manual writing of rules and then uses conditions to judge and classify. The method of manually writing rules and then using conditions to judge and classify usually hard codes the rules and writes them into the system, so that the rules cannot be modified at will, which often leads to the situation that the manual rules are very backward compared to reality. In addition, the method of manually writing rules and then using conditions to judge and classify often requires a lot of manual work, which will greatly reduce work efficiency.

[0006] Traditional machine learning methods do not use the feature extraction layer of deep learning, which makes it impossible to effectively map and classify high-dimensional feature vectors. In the past, when the system complexity was not amazing, it could complete the task well. However, today's system complexity is increasing, and the length of feature vectors has also increased significantly. At this time, traditional machine learning methods cannot cope well.

[0007] Therefore, there is an urgent need for a data interface value evaluation method that can solve the problems of difficulty in writing manual rules and slow updates while ensuring effective rating of the data interface value. Summary of the invention

[0008] 1. Technical issues to be resolved

[0009] The present invention provides a data interface value evaluation method to solve the problems of poor timeliness, low coverage and low efficiency in traditional manual rule construction, overcome the disadvantage of low support dimension of traditional machine learning methods, and can judge and predict the current and future data interface value classification ratings through past data.

[0010] (II) Technical solution

[0011] In order to solve the above technical problems, the present invention provides a data interface value evaluation method based on deep learning, comprising the following steps:

[0012] (1) By collecting various data of the current data interface call, a feature vector of the data interface access status is constructed;

[0013] (2) Input the feature vector constructed in the previous step into the deep learning model, and calculate the output analysis discriminant vector through deep learning technology;

[0014] (3) The analysis discriminant vector is input into a Logistic multi-classification regression model, and the data interface value rating is finally obtained through the Logistic multi-classification regression analysis model.

[0015] The invention also provides a system for implementing the method.

[0016] The present invention also provides a method for constructing a data center based on the method.

[0017] The present invention also provides a method for realizing data service construction based on the method.

[0018] The present invention also provides a method for implementing data operation and maintenance based on the method.

[0019] (III) Beneficial effects

[0020] Compared with the existing rule-based data interface value evaluation method, the present invention uses deep learning technology and Logistic multi-classification regression analysis model in data interface value evaluation, which not only solves the problems of poor timeliness, low coverage and low efficiency of traditional manual rule construction, but also overcomes the disadvantage of low support dimension of traditional machine learning methods, and can judge and predict the current and future data interface value evaluation through past data. The present invention can ensure that the data interface value evaluation is provided under the premise that the interface call data input is correct, while improving the classification performance and accuracy. Therefore, the present invention will play an important role in data interface value evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is the overall flow chart of the method of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0023] The present invention provides a data interface value evaluation method using deep learning technology and Logistic multi-classification regression analysis model in data interface value evaluation. When using the data interface value evaluation method based on deep learning technology and Logistic multi-classification regression analysis model disclosed by the present invention, the system uses a uniformly collected high-dimensional feature vector that can reflect all aspects of data interface information, thereby avoiding complicated manual rule writing and improving efficiency. At the same time, the method of the present invention uses a fully connected deep neural network, which can effectively map and classify high-dimensional data interface access state vectors, thereby better improving the data interface value evaluation rating classification performance. At the same time, the system can modify the rules conveniently and quickly by changing the feature vector table, ensuring the flexibility and expansibility of the system. The structure of the deep model does not need to be greatly changed according to the rules, so it helps to keep the model stable and improve the predictability and stability of the method results.

[0024] The present invention is applied to the evaluation of data interface value, which is helpful for maintaining interface value, optimizing data maintenance costs, reducing the difficulty of data operation and maintenance, etc., and plays a vital role in the construction of enterprise data middle platform and data resource sharing. The use of deep learning technology and Logistic multi-classification regression analysis model not only solves the problems of poor timeliness, low coverage and low efficiency of traditional manual rule construction, but also overcomes the disadvantage of low support dimension of traditional machine learning methods, and can judge and predict the current and future data interface value classification rating through past data.

[0025] To achieve the above purpose, refer to Figure 1 The present invention provides a data interface value evaluation method based on deep learning technology and Logistic multi-classification regression analysis model, comprising the following steps:

[0026] (1) By collecting various data of the current data interface call, a feature vector of the data interface access status is constructed;

[0027] In this embodiment, a feature vector table is first formulated. The feature vector table contains the features of the data interface access state and the corresponding values ​​of the features. The values ​​are discrete integer feature values, and the features of the specific data interface access state are encoded into feature vectors through the feature vector table; each feature vector is assigned the discrete integer feature value. Finally, the data interface access state features at each moment are encoded into a fixed-length feature vector {x1, x2, x3.., x n}, where n is the dimension of the feature vector.

[0028] For example, the following feature vector table is given:

[0029] Table 1 Feature vector table

[0030] feature Value Number of accesses to the interface in the past year Integer Number of accesses to the interface in the past month Integer Number of accesses to the interface in the past day Integer Number of users accessing the interface in the past year Integer Number of users accessing the interface in the past month Integer Number of users accessing the interface in the past day Integer The amount of data provided by the interface in the past year (10,000 records) Integer, less than 10,000 will be rounded up to 10,000 The amount of data provided by the interface in the past month (10,000 records) Integer, less than 10,000 will be rounded up to 10,000 The amount of data provided by the interface in the past day (10,000 records) Integer, less than 10,000 will be rounded up to 10,000 etc…

[0031] The access status of a data interface is: the number of interface accesses in the past year: 256, the number of interface accesses in the past month: 20, the number of interface accesses in the past day: 15, the number of users accessing the interface in the past year: 30, the number of users accessing the interface in the past month: 21, the number of users accessing the interface in the past day: 5, the amount of data provided by the interface in the past year (10,000 records): 512, the amount of data provided by the interface in the past month (10,000 records): 100, the amount of data provided by the interface in the past day (10,000 records): 52. The feature vector constructed for the data interface access status is {256, 20, 15, 30, 21, 5, 512, 100, 52}.

[0032] It is worth noting that the above feature vector table is only an example of the present invention, and the features in the actual production environment need to be dynamically adjusted according to the characteristics of the unit system and the actual performance of the model. The construction of the final feature vector table needs to consider several factors:

[0033] What kind of data interface does your company provide?

[0034] Whether the feature can fully reveal the usage status of the interface;

[0035] Whether the feature is technically discriminative during model building.

[0036] (2) Input the feature vector constructed in the previous step into the deep learning model, and calculate the output analysis discriminant vector through deep learning technology;

[0037] In this embodiment, the feature vector {x1, x2, x3.., x n} is input into a fully connected deep learning model. The structure and hierarchy of the deep learning model can be adjusted according to the actual training and production environment application effects. The deep learning model has a fully connected node structure. The dimensional structure of the deep learning model is n-l1-l2-l3-m, where n is the feature vector {x1,x2,x3..,x n}, m is the dimension of the output analysis discriminant vector, l1, l2, l3 are the intermediate dimensions of the deep learning model, which can be adjusted according to the actual model performance. The feature vector {x1, x2, x3.., x n}After being input into the deep learning model for calculation, an m-dimensional output vector {a1, a2, a3, ...a m}, after the output vector is normalized by softmax, the analysis discriminant vector {b1,b2,b3,…b m}, where b i ∈[0,1].

[0038] For example, the feature vector for constructing the data interface access status is {256, 20, 15, 30, 21, 5, 512, 100, 52}, and the dimension of the feature vector is 9. A fully connected deep learning model has a fully connected node structure. The dimension structure of the deep learning model is 9-50-100-50-10. Input the user's feature vector into the deep model for calculation. Output a 10-dimensional output vector {1.5, 20.1, 0.54, 23.12, 8.23, 101.23, 34.45, 0.023, 32, 28}. After the output vector is normalized by softmax, the classification discriminant vector {0.0807, 0.854, 0.134, 0.986, 0.535, 0.211, 0.885, 0.949, 0.323, 0.454} can be output. This classification discriminant vector will be used as the main discriminant vector for the next step of regression (the numbers in the vector are only for illustration).

[0039] (3) Inputting the analysis discriminant vector into a Logistic multi-classification regression model, and finally obtaining a data interface value rating through the Logistic multi-classification regression analysis model;

[0040] In this embodiment, the analysis and discrimination vector {b1, b2, b3, ... b m} is input into the logistic multi-classification regression analysis model known in the art, and the logistic multi-classification regression analysis model analyzes the input discriminant vector {b1, b2, b3, ... b m}Give the value evaluation classification c of the corresponding data interface, where c∈[0,1,2,3…j], and j is the classification rating of the data interface value in the process of training the deep learning model and the logistics multi-classification regression model.

[0041] For example: the classification discriminant vector {0.0807, 0.854, 0.134, 0.986, 0.535, 0.211, 0.885, 0.949, 0.323, 0.454} of the single interface value is input into the logistic multi-classification regression analysis model, and the logistic multi-classification regression analysis model classifies the input analysis discriminant vector and gives the classification c=3.

[0042] In this embodiment, the specific classification table is shown in Table 2 (the specific classification table needs to be redesigned by the relevant user according to the actual business situation, and this classification table is just an example):

[0043] Table 2 Classification table

[0044] Data interface value evaluation and grading Classification Number The value is extremely high, additional resources are recommended. 1 The value is high, it is recommended to pay attention to it. 2 The value is average, it is recommended to keep it. 3 The value is low, please pay attention. 4 It has no value, it is recommended to be removed from the shelves. 5 etc. …

[0045] The data interface value evaluation classification is c=3, which is of average value and is recommended to be retained.

[0046] It can be seen that the present invention is applied to the evaluation of the value of data interfaces, using an objective method to evaluate the use value of each data interface released by the system, so as to better discover more valuable interfaces for promotion, increase data availability and data asset value. The use of deep learning technology and Logistic multi-classification regression analysis model not only solves the problems of poor timeliness, low coverage and low efficiency of traditional manual rule construction, but also overcomes the disadvantage of low support dimension of traditional machine learning methods, and can judge and predict the current and future data interface value through past data. The present invention has a wide range of applications in the fields of data governance, data warehouse construction and data middle platform.

[0047] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A data interface value evaluation method based on deep learning, characterized in that: The following steps are involved: (1) By collecting various data of the current data interface call, a feature vector of the data interface access status is constructed; (2) Input the feature vector constructed in the previous step into the deep learning model, and calculate the output analysis discriminant vector through deep learning technology; (3) The analysis discriminant vector is input into a Logistic multi-classification regression model, and the data interface value rating is finally obtained through the Logistic multi-classification regression analysis model.

2. The method according to claim 1, characterized in that In step (1), a feature vector table is prepared, wherein the feature vector table contains features of the data interface access state and values ​​corresponding to the features, wherein the values ​​are discrete integer feature values, and the features of a specific data interface access state are encoded into feature vectors through the feature vector table; Each feature vector is assigned the discrete integer feature value; finally, the data interface access state feature at each moment is encoded into a fixed-length feature vector {x1, x2, x3.., x n }, where n is the dimension of the feature vector.

3. The method according to claim 2, characterized in that In step (2), firstly, the feature vector {x1,x2,x3..,x n } is input into a deep learning model, which has a fully connected node structure. The dimensional structure of the deep learning model is n-l1-l2-l3-m, where n is the feature vector {x1,x2,x3..,x n }, m is the dimension of the output analysis discriminant vector, l1, l2, l3 are the intermediate dimensions of the deep learning model, and the feature vector {x1, x2, x3.., x n }After being input into the deep learning model for calculation, an m-dimensional output vector {a1, a2, a3, ... a m }, the output vector is normalized by softmax, and the output analysis discriminant vector {b1,b2,b3,…b m }, where b i ∈[0,1].

4. The method according to claim 3, characterized in that The structure and hierarchy of the deep learning model are adjusted according to the actual training and production environment application effects.

5. The method according to claim 3, characterized in that In step (3), the analysis discriminant vector {b1, b2, b3, ... b m } is input into the logistic multi-classification regression analysis model, and the logistic multi-classification regression analysis model analyzes the input discriminant vector {b1, b2, b3, …b m }Give the value evaluation classification c of the corresponding data interface, where c∈[0,1,2,3…j], and j is the classification rating of the data interface value in the process of training the deep learning model and the logistics multi-classification regression model.

6. The method according to claim 5, characterized in that The gradient descent method is used to train the deep learning model and the logistics multi-classification regression model.

7. A system for implementing the method according to any one of claims 1 to 6.

8. A method for building a data middle platform based on the method described in any one of claims 1 to 6.

9. A method for implementing data service construction based on the method described in any one of claims 1 to 6.

10. A method for implementing data operation and maintenance based on the method described in any one of claims 1 to 6.

Images