Big data intelligent processing neural network credibility modeling method

Abstract

According to the credibility modeling method for the big data intelligent processing neural network, the credibility model is established for the identification result of the artificial neural network by using the artificial neuron output layer, so that the credibility model can describe the correct probability of the classification result, that is, the correctness of the classification result is considered through the credibility, and the accuracy of the classification result is improved. The probability degree trust the classification is obtained. Mathematical modeling is carried out by using a neuron output layer, so that a neural network credibility model is successfully established and is used for evaluating a single recognition result of a classifier. On the basis, a traditional method is further improved, a multi-output neural network, a single-classifier rejection algorithm of the credibility of a single-output neural network and a multi-classifier fusion algorithm of the multi-output neural network are established for neural networks of different output types, the credibility evaluation capability of the neural networks is high, the credibility calculation accuracy is high, and the reliability of the neural networks is improved. The neural network is good in reliability, application safety and effectiveness.

Description

Technical Field

[0001] The present application relates to a big data neural network credibility modeling method, and in particular to a big data intelligent processing neural network credibility modeling method, belonging to the technical field of neural network credibility modeling. Background Art

[0002] In today's world, information technology is advancing at an unimaginable pace. The amount of data generated daily is enormous and growing at an exponential rate, marking the beginning of the era of big data. Faced with such a vast and complex amount of information, relying solely on the human brain to identify and interpret it is clearly unrealistic. Therefore, how to effectively and correctly use computers to process this information instead of the human brain has become a hot topic in research and development. Against this backdrop, artificial neural networks, with their powerful data processing capabilities and highly intelligent self-learning abilities, have garnered significant attention from the academic community and are now widely used in various fields.

[0003] The field of artificial neural network research and development is flourishing. To adapt to diverse application problems, developers have constructed a variety of network architectures, but in essence, they all mimic the neural network architecture of the human brain. Developers test the neural network using a large number of learning samples and then use the tested network to make predictions on additional test data, reflecting the learning ability of the human brain. In previous research and development, the concept of "recognition rate" was often used to evaluate neural network prediction results. The recognition rate refers to the percentage of correct predictions in a certain number of test sets. However, calculating the recognition rate requires conducting a large number of test experiments to evaluate the neural network classifier based on these calculations. This does not include the evaluation of individual prediction results. The results of individual predictions are uncertain. Even if the classifier has a high overall recognition rate, it cannot guarantee that the classification is completely correct. This can lead to misjudgment of critical data and cause serious problems. Furthermore, in production, test samples are often unlabeled, noisy, and spatially unevenly distributed. This results in: for test samples, the error is generally small, and convergence is fast; however, for validation samples, the error may be large, and convergence is relatively slow. Once a network model based on test samples is established, the key issue becomes determining the degree of confidence in the neural network model's (classifier's) predictions for each test sample.

[0004] The problems that need to be solved in existing big data neural network credibility modeling and the key technical difficulties of this application include:

[0005] (1) The artificial neural network classifiers in the prior art only use the output layer to obtain a classification result, while ignoring other information that may be contained in the output layer. There is a lack of using the artificial neuron output layer to establish a credibility model for the artificial neural network recognition result, so that it can describe the correct probability of the classification result. It is impossible to consider the correctness of this classification result through credibility, and it is impossible to determine the degree of confidence in this classification. There is a lack of mathematical modeling using the neuron output layer, and no neural network credibility model is established and used to evaluate the single recognition result of the classifier. There is a lack of improvement on traditional methods, and no single classifier rejection algorithm for the credibility of multi-output neural networks and single-output neural networks and a multi-classifier fusion algorithm for multi-output neural networks are established for neural networks with different output types. It is impossible to efficiently solve the problem of big data neural network credibility modeling and cannot evaluate the credibility calculation of big data neural networks, which restricts the application of neural networks.

[0006] (2) The essence of existing artificial neural networks is to imitate the neural network architecture of the human brain. When evaluating the prediction results of neural networks, the concept of "recognition rate" is often used. The recognition rate refers to the percentage of the number of correct prediction results in a certain number of test sets to the number of tests. However, the premise of calculating the recognition rate is that a large number of test experiments must be carried out to evaluate the neural network classifier based on the calculation. However, this does not include the evaluation of the results of a single prediction. The results of a single prediction are uncertain. Even if the overall recognition rate of the classifier is very high, it cannot be completely guaranteed that the classification is completely correct. This may lead to misjudgment of key data and cause major problems. Moreover, in production activities, test samples are often unlabeled, and the test sample data is noisy and unevenly distributed in space. The result is that for test samples, the error obtained is generally small and the convergence speed is relatively fast; however, for test samples, the error obtained may be relatively large and the convergence speed is relatively slow. Once the network model obtained based on the test sample test is established, the key point lies in how to determine the degree of trustworthiness of the neural network model (classifier) ​​in each prediction of the test sample. Existing technologies cannot solve these problems well, resulting in poor credibility evaluation ability of the neural network, low accuracy of credibility calculation, poor reliability of the neural network, and poor application security and effectiveness.

[0007] (3) The accuracy of artificial neural networks is increasing, but the research and development of neural network credibility is not perfect. The concept of credibility has not been introduced for deep neural networks. The existing technology lacks a method to use the output layer of the network model itself to build a credibility model. It is impossible to measure the credibility of a single prediction and cannot provide a reference for the evaluation of the output results. There is a lack of classifiers for different output types. There is a lack of two experiments representing multi-output neural networks and single-output neural networks in handwritten character recognition and face gender recognition. It is impossible to perform credibility modeling and effect testing for each. The accuracy and reliability of neural network credibility recognition are poor. It is impossible to use credibility to fuse multiple classifiers to obtain more reliable results. The credibility fusion recognition accuracy is negatively affected by the low recognition rate classifier and cannot maintain the highest recognition accuracy level among the original classifiers. The accuracy of neural network credibility calculation is poor.

[0008] (4) The accuracy of the artificial neural network model is determined by the network architecture level used, the initial parameter settings, the settings of the hidden layer nodes, the type of activation function used, the type of learning algorithm used, and the quality of the test samples. Artificial neural networks have a huge demand for samples. During the model establishment phase, a large number of test samples are needed to test the network. During the test phase, a large number of test samples are also needed to test the performance of the model. Test samples are used to test the established network and are divided into test samples with known outputs and test samples without known outputs. In actual production, test samples often do not have known outputs. The quality of the test sample data affects the effect of the network model to a certain extent. If the test samples used have large noise or the number of samples in each category is not evenly distributed, then the network model obtained in this case is not accurate and there will be an overfitting problem. Specifically, the model fits the test data very well, but the test sample has a poor effect. That is, the artificial neural network model has a good memory ability but a poor prediction ability. In order to improve the predictive ability of artificial neural networks, we must pay attention to increasing the data of test data and improving its quality, and through repeated testing, in order to achieve better results. However, after the test is completed and the network model has been established, how to predict the credibility of the prediction output results of the artificial neural network model for non-test samples becomes an urgent problem to be solved. Summary of the Invention

[0009] In response to the problems of the prior art, this application provides a method for constructing a credibility model using the output layer of the network model itself, measuring the credibility of a single prediction, and providing a reference for the evaluation of the output results from another aspect. This application uses convolutional neural networks, and for classifiers of different output types, uses handwritten character recognition and face gender recognition as two experiments to represent multi-output neural networks and single-output neural networks, respectively, to perform respective credibility modeling and effect testing. The experiments show that the improved algorithm significantly improves the accuracy and reliability of recognition on the basis of low consumption. In addition, this application also considers how to use credibility to fuse multiple classifiers to obtain more reliable results. Experiments show that the credibility fusion of multiple classifiers not only ensures that the recognition accuracy after fusion is not negatively affected by the low recognition rate classifier, and maintains the highest recognition accuracy level in the original classifier, but also can slightly improve the recognition accuracy at this level, thereby achieving an optimized recognition effect.

[0010] To achieve the above technical effects, the technical solutions adopted in this application are as follows:

[0011] This method uses a neural network credibility modeling approach for intelligent processing of big data. Starting from evaluating a single prediction result of a classifier, it uses the output layer of the network model itself to construct a credibility model for a deep neural network, measuring the trustworthiness of a single prediction. Based on a convolutional neural network, it uses handwritten character recognition and facial gender recognition as multi-output neural networks and single-output neural networks, respectively, for classifiers with different output types, conducting credibility modeling and effectiveness testing. Furthermore, credibility is used to fuse multiple classifiers to obtain more reliable results.

[0012] 1) Credibility modeling of big data neural networks: This includes the credibility modeling of multi-output neural networks, the credibility modeling of single-output neural networks, and the intelligent fusion of multiple classifiers. The credibility modeling of multi-output neural networks is divided into classification based on normalized credibility index and intelligent output based on binary. Credibility modeling of big data neural networks establishes mathematical models of the credibility of deep neural networks under two output modes: multi-output classifiers and single-output classifiers. In addition, the credibility fusion of multiple classifiers with multiple outputs is modeled to construct a deep neural network classification algorithm that takes credibility into consideration.

[0013] 2) Deep Learning Credibility Fusion: This includes handwriting recognition network architecture, facial gender recognition network architecture, single-classifier credibility rejection of multi-output classifiers, single-classifier credibility rejection of single-output classifiers, and multi-classifier credibility fusion of multi-output classifiers. Through credibility-based handwriting recognition, facial gender recognition, and credibility fusion methods, a credibility model for evaluating the effectiveness of a prediction result is established. The credibility fusion of multiple classifiers is superior to all original classifiers.

[0014] Preferably, based on normalized credibility index classification: a multi-output neural network sets N output nodes in the output layer, each representing N categories. Each forward propagation output result obtains an N-dimensional array, and the size of each dimension value in the array represents the probability of the corresponding category.

[0015] The probability of each event occurring is between 0 and 1. The classification problem is considered a probabilistic event, where a sample belongs to a certain category. The range of each dimension of the N-th array is set to [0, 1]. Normalized confidence index classification helps the learning algorithm optimize the classification results. A normalized confidence index layer is added at the end of the neural network to convert the output into a probability distribution.

[0016] Assume that the initial neural network output is y1, y2, ..., yn, and the output is obtained after normalized credibility index classification processing. The credibility model is established based on the output array after normalized credibility index classification. The modeling method is as follows:

[0017] Credibility is determined by the following two components: one is significance, denoted as P r , the other is resolution, denoted as P d , significance P r Is the component representing the classification result as the maximum item in the output vector large enough? The output vector is defined as a vector with each neural network output as a component. In the normalized credibility index output mode, the components of the output vector are not greater than 1 and not less than 0. The significance value is represented by the maximum item value in the output vector, as shown in Formula 1. The resolution P d Whether the component representing the classification result is sufficiently prominent compared with other components in the output vector. The mathematical expression of resolution is expressed as the value of the maximum term divided by the sum of all terms, as shown in Equation 2;

[0018] P r =O m Formula 1

[0019]

[0020]

[0021] Whether the classification result is significant and whether the classification result is discriminative. The occurrence of these two events does not affect each other and is independent of each other. The probability of independent events occurring simultaneously is calculated by multiplying the probabilities of the two events occurring separately. Then, under the classification of the neural network, the probability that the classification result is correct, that is, the calculation formula of the reliability C is shown in Formula 3. Let the output vector be O m Indicates the item with the largest component value in the output vector, 0 tRepresents any component in the output vector, and Equation 3 can be used to calculate the credibility of each prediction under the maximum output.

[0022] Preferably, the intelligent output is based on binary: the output categories are numbered in a binary coding manner, and the dimension of the output vector is reduced when there are many classification categories. The dimension of the result vector is the number of bits of the binary number. The component value range of the output result vector should be [0, 1]. If the value of a component of the result vector is close to 0, it means that the binary number takes 0 at that bit; if the value of a component of the result vector is close to 1, it takes 1, and then the binary number is converted into a decimal number to obtain the actual classification category;

[0023] Credibility modeling in binary output mode is determined by saliency and resolution, and the output vector is transformed before calculation;

[0024] Suppose there are B outputs after encoding, that is, the output result represents a B-bit binary number, and the output vector is 0 i for The i-th component of a binary digit has two possible results: 0 or 1. The output vector is transformed. The transformed output vector is O′ i for The i-th component of m for The component with the largest median value, 0 i and O′ i The transformation relationship is shown in formula 4, where b i Represents the number on the i-th position of the binary number of this category when the conversion is correct, b i The value is 0 or 1, according to the vector Get the significance P in binary mode r and resolution P d , the results are shown in Equation 5 and Equation 6, and finally the credibility model in binary mode is obtained, as shown in Equation 7:

[0025] O′ i =|O i -(1-b i )| Equation 7

[0026] The steps of the single classifier rejection algorithm of the multi-output neural network with added credibility are as follows:

[0027] Step 1: Determine the parameters of the classifier, including the number of inputs, outputs, and intermediate layers;

[0028] Step 2: Test the classifier using a large number of test samples until the requirements are met;

[0029] Step 3: Use the test sample to test, each test sample will get an output vector and a credibility;

[0030] Step 4: Evaluate the classification results according to the credibility C and set the critical value C T , if C≥C T , then adopt the classification result, if C<C T , then reject the classification result.

[0031] Preferably, the credibility modeling of a single-output neural network is as follows: a single-output neural network has only one output node in the output layer, and the output value is between [0, 1]. A value close to 0 is judged as yes / no, and a value close to 1 is judged as no / yes.

[0032] To establish a credibility model for a single-output neural network, we first use parallel output. Assuming the existence of a multiverse containing multiple world lines, each world line exists independently and equivalently. Parallel output is to establish multiple outputs that are equivalent to but different from the original output to form a set containing multiple outputs. The result of the original output being transformed by a certain function is used as a parallel output. This function is defined as the T function, which must meet the following requirements:

[0033] (1) Reversible, that is, the T function has an inverse function T-1 to perform reverse calculation from parallel output to original output. Let the original output be y and the parallel output be y t , then y t =Z(y),y=T -1 (y t ), T and T -1 They are inverse functions of each other;

[0034] (2) Continuous, that is, T function y t =T(y) in the rectangular coordinate system is a continuous curve without breaks. t The change of changes continuously with the change of the independent variable y;

[0035] (3) Monotonic, that is, in the domain of definition, the T function is monotonic, monotonically increasing or monotonically decreasing. If y1≥y2, then y t1 ≥y t2 (or y t1 ≤y t2 );

[0036] The function that meets the above conditions is called T-function. T-function ensures that there is a close relationship between the parallel output and the original output. Through a series of T-functions, multiple outputs parallel to the original output are quickly established. The credibility modeling of a single output is based on the original output and these parallel outputs.

[0037] T functions include: 1) 2) 3)y t =e y ;

[0038] The modeling method is as follows:

[0039] (1) When establishing the output layer of the neural network, N more components are added, and the expected value of the component is the T transformation of the original expected value. Assume that the set of T functions used is {T 1 , T 2 ,…,T i , T N}, then the expected value of the output vector is

[0040]

[0041] (2) Test the modified network and get multiple output vectors.

[0042]

[0043] (3) Perform the corresponding inverse transformation of T transformation on the output components except the first term, and obtain {y′, y 1′ ,y 2′ ,…,y i′ ,…,y N′};

[0044] (4) Solve {y′, y 1′ ,y 2′ ,…,y i′ ,…,y N′}'s variance D;

[0045] (5) Credibility C = 1-D;

[0046] The steps of the single classifier rejection algorithm of the single-output neural network with added credibility are as follows:

[0047] Step 1: Determine the parameters of the classifier, including the number of inputs, outputs, and intermediate layers;

[0048] Step 2: Determine the T function to be used and add parallel output to the network output layer;

[0049] Step 3: Adjust the sample labels to adapt to the network with increased parallel output.

[0050] Step 4: Test the classifier using a large number of test samples until the requirements are met;

[0051] Step 5: Use the test samples to test. Each test sample will get a set of outputs, and a credibility is calculated.

[0052] Step 6: Evaluate the classification results based on the credibility C and set the critical value C T , if C≥C T , then adopt the classification result, if C<C T , then reject the classification result.

[0053] Preferably, the credibility of multiple classifiers is intelligently integrated: in the maximum output mode, the output vector Defined as a vector with each neural network output as a component. In binary mode, the output vector refers to the transformed output vector. The output vectors of classifiers with different output modes have different meanings and cannot be fused.

[0054] Formula 8 obtains the fused result vector, where n represents a total of n classifiers. represents the output vector of the i-th classifier, C i is the classification credibility of the i-th classifier in this prediction, Denotes the adjusted output vector obtained after credibility fusion, let O′ cj express The jth component of cm express The largest item in the component is the fusion significance P′ r and discrimination P′ d As shown in Equation 9 and Equation 10 respectively, multiply the two to obtain the fused credibility C′, as shown in Equation 11, C′ is the credibility of the fused n classifiers;

[0055]

[0056] P′ r =O′ cm Formula 9

[0057]

[0058]

[0059] When performing multi-classifier recognition on a series of samples, the multi-classifier fusion using credibility can obtain recognition results that are better than those of a single classifier. The process steps of the multi-classification credibility fusion algorithm are as follows:

[0060] Step 1: Use n different classifiers to classify the same sample, and obtain n sets of output vectors and n credibility values;

[0061] Step 2: Weight the credibility of these n groups of output vectors according to the established model to obtain a fused output vector;

[0062] Step 3: Get the classification result based on the fused output vector.

[0063] Preferably, the handwriting recognition network architecture uses a convolutional neural network to test and learn handwriting fonts, and adopts a maximum output mode for verification. The input of the convolutional neural network used is a handwritten digital image of 28×28 pixels. Through feature extraction and downsampling, an output vector with 10 components is finally obtained.

[0064] The network architecture consists of a data input layer, convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, ReLU layer, fully connected layer 2, and a normalized confidence index layer. The initial input image size is 32×32 pixels, where C represents the convolutional layer, S represents the subsampling layer, and F represents the fully connected layer. The first C1 convolutional layer uses 6 convolution kernels in a single-channel model to obtain 6 feature maps, where the size of each convolution kernel is 5×5. The initial input image is convolved with the convolution kernel, and the resulting feature map size is (32-5+1)×(32-5+1)=28×28. The number of parameters used in this process is (5×5+1)×6=156, and the number of connections is (5×5+1)×28×28×6=122304, where 5×5 is the convolution template parameter, 1 is the bias parameter, and 28×28 is the size of the image after convolution.

[0065] Subsampling layer 1-S2 uses 6 feature maps, each of which is 14×14 in size. The hidden unit of each feature map is connected to the 2×2 unit of the feature map corresponding to the previous layer C1. The calculation process is: the values ​​in the 2×2 unit are added and then multiplied by the weight w, plus a bias parameter b. Each feature map shares the weight, and then uses the activation function to process it, and the obtained value is used as the value of the corresponding unit. The side length of each feature map in the S2 layer is half of that of the previous layer C1. The S2 layer requires 2×6=12 parameters and the number of connections is (4+1)×14×14×6=5880.

[0066] Convolutional layer 2-C3 uses 14 channels and 16 convolution kernels. The size of the convolution kernel is 5×5. C3 is partially connected to the previous layer S2. C3 includes 16 feature maps. The size of each feature map is (14-5+1)×(14-5+1)=10×10. Each feature map is only connected to part of the feature maps in the previous layer S2. The calculation process is described with the 0th feature map: 1 convolution kernel is used to perform convolution calculations with the 3 feature maps of the S2 layer respectively, and then the convolution results are added, plus a bias, and then processed by the activation function to obtain the corresponding feature map. The number of parameters required is (5×5×3+1)×6+(5×5×4+1)×9+5×5×6+1=1516, where 5×5 is the convolution parameter. The convolution kernel has 3, 4, and 6 convolution templates respectively, and the number of connections is 1516×10×10=151600.

[0067] Subsampling layers 2-S4 set 16 feature maps, each feature map has a size of 5×5, the number of parameters is 16×2=32, and the number of connections is (4+1)×5×5×16=2000;

[0068] Convolutional layer 3-C5 sets 120 convolution kernels, the size of the convolution kernel is 5×5, including 120 feature maps, the convolution kernel has 16 convolution templates, the size of the feature map is 1×1, just becoming fully connected, the C5 layer has 120×(5×5×16+1)=48120 parameters;

[0069] The fully connected layer F6 has 86 neurons, each of which is fully connected to C5. The number of connections and parameters are both 86×120=10164. The F6 layer obtains an 86-dimensional feature, which is subsequently used for classification prediction.

[0070] Preferably, the face gender recognition network architecture: the network input data uses 3 color channels, the initial image size is 816×816, the size is adjusted to 256×256, and then the adjusted image is randomly cropped to a size of 224×224 after cropping, and used as the input of the network;

[0071] Cascaded 3×3 convolution kernels are used as the underlying feature extraction, and edge expansion is adopted. The receptive field of three cascaded 3×3 convolution kernels is 7×7, the weight parameters of the 7×7 convolution kernel are 7×7×64=49×64, and the weight parameters of the cascaded small-size convolution kernel are 3×3×3×64=27×64. The fusion of middle-level features and high-level features is achieved by crossing the convolution layer. The L9 and L10 layers perform complex calculations in each local receptive field to extract various potential nonlinear features. Lll_a uses global average pooling, and the L7 layer spans L8 to 10 layers and is directly fully connected to the last hidden layer. The BN layer is added to play a normalization role.

[0072] Preferably, the single classifier credibility rejection of the multi-output classifier: credibility modeling is performed on the multi-output neural network, the handwritten character recognition network model has multiple outputs, and the handwritten digit recognition output components have 10, representing the 10 digits 0-9 respectively;

[0073] To build a detection model, the steps are as follows:

[0074] Step a: Establish multiple classifiers by adjusting the number of iterations. In the maximum output mode, the iterations are 10000, 9000, 8000, 7000, 6000, 5000, 4000, 3000, 2000, and 1000 respectively. The obtained classifiers are numbered as classifier 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10.

[0075] Step b: Test the above ten classifiers using the same test samples, calculate the credibility of the classification results, and calculate and analyze the different distributions of the credibility values ​​in the cases of correct and incorrect classification;

[0076] Step c: Set the credibility threshold for the classifier, and analyze the changes in the recognition accuracy when setting different critical values. The recognition error rate and changes of each classifier are analyzed when there is no critical value and when the critical value is set to 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, and 0.50. The recognition error rate is the total test probability that the classification result of the test sample in the classifier does not match the label after removing the examples with credibility lower than the critical value.

[0077] Preferably, the single-output classifier credibility rejection is used: face gender recognition is used to verify the credibility modeling effect of the single-classification output network, and the network output components are changed from two to one. The judgment standard of the single output is: if the result is close to 0, it is judged as male, and if the result is close to 1, it is judged as female. This does not conflict with the original design purpose of the network and can meet the single-output network requirement. Therefore, this method is used to modify the network. The specific steps are as follows:

[0078] Step I: Different classifiers are obtained by adjusting the number of iterations. The same test set is tested. Five classifiers are obtained by iterating 10,000, 15,000, 20,000, 25,000, and 30,000 times, respectively. They are numbered as classifiers A, B, C, D, and E.

[0079] Step II: For the five classifiers obtained above, three parallel outputs are added to the original network single output and modeled. The T functions used are y t =y 2 ;y t =e y;

[0080] Step III: Use the same test samples to test classifiers A, B, C, D, and E, record their respective accuracy rates and calculated credibility values, and calculate and analyze the different distributions of credibility values ​​in correct and incorrect classification cases.

[0081] Preferably, the credibility fusion of multiple classifiers of a multi-output classifier: the output modes of the classifiers are all maximum outputs, and the steps are as follows:

[0082] Step A: Change the initial weights of the neural network and iterate 1000 times to obtain 5 classifiers, numbered I, II, III, IV, and V. Use the same test samples to test these 5 classifiers and obtain the recognition accuracy of each classifier;

[0083] Step B: Use the model established above to fuse the above five classifiers, numbered V, and test the same test samples to obtain the recognition accuracy after fusion;

[0084] Step C: Change the number of iterations and repeat the operations of steps 1 and 2 to obtain groups with the number of iterations being 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, and 10000;

[0085] Step 4: Compare the recognition accuracy of the five classifiers in the same group with the recognition accuracy of the fusion classifier.

[0086] Compared with the existing technology, the innovation and advantages of this application are:

[0087] (1) The traditional artificial neural network classifier uses the output layer only to obtain a classification result, while ignoring other information that the output layer may contain. The present application uses the artificial neuron output layer to establish a credibility model for the result of artificial neural network recognition, so that it can describe the correct probability of the classification result, that is, to consider the correctness of the classification result through credibility, and obtain the degree of probability of trusting the classification. By using the neuron output layer for mathematical modeling, the present application successfully established a neural network credibility model and used it to evaluate the single recognition result of the classifier. On this basis, the traditional method is further improved, and a single classifier rejection algorithm for the credibility of multi-output neural networks and single-output neural networks and a multi-classifier fusion algorithm for multi-output neural networks are established for neural networks with different output types. The neural network has strong credibility evaluation capabilities, high credibility calculation accuracy, good neural network reliability, and good application security and effectiveness.

[0088] (2) This application takes the evaluation of a certain prediction result of the classifier as the starting point, uses the output layer of the network model itself to construct a credibility model for the deep neural network, and measures the credibility of a single prediction; based on the convolutional neural network, for classifiers of different output types, handwritten character recognition and face gender recognition are respectively multi-output neural networks and single-output neural networks, and their respective credibility modeling and effect testing are carried out; in addition, credibility is used to fuse multiple classifiers to obtain more reliable results. The improved algorithm significantly improves the accuracy and reliability of recognition on the basis of low consumption. The establishment of the above credibility model provides a practical, effective and simple channel for evaluating the credibility of a single recognition result. Furthermore, the single classifier rejection based on credibility can exclude recognition results with a high probability of error, thereby improving the accuracy rate; the classification result obtained by the fusion of multiple classifiers based on credibility is higher than the accuracy rate of any original classifier, which greatly improves the recognition effect. At the same time, the recognition algorithm based on credibility has the advantages of simple calculation and fast speed. It can be implemented with only a small amount of calculation. This method has very important theoretical value and engineering application value.

[0089] (3) In response to the problems of the prior art, this application provides a method for constructing a credibility model using the output layer of the network model itself, measuring the credibility of a single prediction, and providing a reference for the evaluation of the output results from another aspect. This application uses convolutional neural networks, and for classifiers of different output types, uses handwritten character recognition and face gender recognition as two experiments to represent multi-output neural networks and single-output neural networks, respectively, to perform respective credibility modeling and effect testing. The experiments prove that the improved algorithm significantly improves the accuracy and reliability of recognition on the basis of low consumption. In addition, this application also considers how to use credibility to fuse multiple classifiers to obtain more reliable results. Experiments show that the credibility fusion of multiple classifiers not only ensures that the recognition accuracy after fusion is not negatively affected by the low recognition rate classifier, and maintains the highest recognition accuracy level in the original classifier, but also can slightly improve the recognition accuracy at this level, thereby achieving an optimized recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 This is a flow chart of the single classifier rejection algorithm of a multi-output neural network with added credibility.

[0091] Figure 2 This is a flow chart of the single classifier rejection algorithm of a single-output neural network with added credibility.

[0092] Figure 3 This is the flowchart of the credibility fusion algorithm for multi-classification.

[0093] Figure 4 It is a statistical diagram of the credibility calculation of each classifier test of the multi-output classifier.

[0094] Figure 5 It is a statistical diagram of the credibility calculation of each classifier test for a single-output classifier.

[0095] Figure 6 This is a schematic diagram comparing the recognition accuracy of the classifiers before and after fusion. DETAILED DESCRIPTION

[0096] The following, in conjunction with the accompanying drawings, further describes the technical solution of the big data intelligent processing neural network credibility modeling method provided by this application, so that those skilled in the art can better understand this application and implement it.

[0097] In response to the above problems, this application provides a method for constructing a credibility model using the output layer of the network model itself, measuring the credibility of a single prediction, and providing a reference for the evaluation of the output results from another aspect. This application takes convolutional neural networks as an example, and for classifiers of different output types, uses handwritten character recognition and face gender recognition as two experiments to represent multi-output neural networks and single-output neural networks, respectively, to conduct respective credibility modeling and effect testing. The experiments prove that the improved algorithm significantly improves the accuracy and reliability of recognition on the basis of low consumption. In addition, this application also considers how to use credibility to fuse multiple classifiers to obtain more reliable results. Experiments show that the credibility fusion of multiple classifiers not only ensures that the recognition accuracy after fusion is not negatively affected by the low recognition rate classifier, and maintains the highest recognition accuracy level in the original classifier, but also can slightly improve the recognition accuracy at this level, thereby achieving an optimized recognition effect.

[0098] 1. Credibility Modeling of Big Data Neural Networks

[0099] The accuracy of an artificial neural network model is determined by the network architecture level used, the initial parameter settings, the hidden layer node settings, the type of activation function used, the type of learning algorithm used, and the quality of the test samples. Artificial neural networks have a high demand for samples. During the model establishment phase, a large number of test samples are needed to test the network. During the testing phase, a large number of test samples are also needed to test the model's performance. Test samples are used to test the established network and are divided into test samples with known outputs and test samples without known outputs. In actual production, test samples often do not have known outputs. The quality of the test sample data affects the effectiveness of the network model to a certain extent. If the test samples used are noisy or the number of samples in each category is not balanced, the resulting network model will be inaccurate and will suffer from overfitting. Specifically, the model fits the test data well but performs poorly on the test samples. In other words, the artificial neural network model has good memory but poor predictive ability. In order to improve the predictive ability of artificial neural networks, we must pay attention to increasing the data of test data and improving its quality, and through repeated testing, in order to achieve better results. However, after the test is completed and the network model has been established, how to predict the credibility of the prediction output results of the artificial neural network model for non-test samples becomes an urgent problem to be solved.

[0100] The credibility value range is C∈[0,1]. The larger the value, the higher the reliability of the experimental results. The credibility introduced by this application for the neural network is not for the classifier itself, but for a certain prediction result. The credibility is applicable to evaluating this prediction result, not to evaluating the classifier.

[0101] To calculate the credibility value of a deep neural network, we first establish a mathematical model for credibility calculation. Depending on the classifier output settings, there are two output modes: multi-output and single output. Multi-output is further divided into two modes: maximum output and binary output. Maximum output refers to finding the component with the largest value in the result vector, and the class represented by this component is the result class; binary output encodes all categories in binary, and each component of the result vector corresponds to each digit of the binary number. The digit is 0 or 1 based on the value of the component. Finally, the corresponding category is found based on the resulting binary number.

[0102] (1) The significance of credibility modeling

[0103] In today's information-exploding world, obtaining reliable information from a vast amount of information is crucial. Similarly, judging and evaluating the different classification results of various classifiers to obtain the desired results requires analyzing the credibility of this classification information. In traditional artificial neural network applications, only a set of result vectors is obtained at the output, without a quantitative reference for the reliability of the results. Introducing the concept of credibility allows for simultaneous assessment of the reliability of classification results, providing a preliminary assessment of their accuracy, which is of great significance in practical applications.

[0104] (1) Multi-classifier fusion. In actual operations, in order to increase the accuracy of classification results, multiple classifiers are often used to classify the same sample. In this scenario, it is inevitable that the classification results of the classifiers will be different. If the classification effect is simply calculated by averaging all the classification results, the classification effect will be worse than that of using a single classifier, which runs counter to the original intention of using multiple classifiers. If the credibility is used as a weight and the results of all classifiers are weighted and summed on this basis, the result will be more accurate.

[0105] (2) Cluster credibility fusion. When judging the credibility of a cluster with a certain characteristic, the credibility of the cluster with this characteristic is calculated by the credibility of each point with this characteristic. For example, in face recognition, to judge the credibility of the presence of a face in a certain area, the credibility of the presence of a face can be judged in each component of the image. The credibility of the presence of a face in the area is obtained by multiplying the result of 1 minus all unreliable values ​​(1 minus credibility). In this way, the credibility of each component in the cluster can be fused into the credibility of the entire cluster.

[0106] (3) Implementation of the tracking and monitoring system. When tracking and identifying images, the following method is used: the entire image is traversed to search for the target on the initial image. After the target position is found, the target is searched for in the subsequent images starting from the vicinity of the reference position based on this position. If the target is not found nearby, the entire image is traversed. This method greatly improves work efficiency compared to traversing and searching on each image. However, this tracking and monitoring method has a disadvantage: if the target is recognized incorrectly at the beginning, the subsequent images will not be able to find the target starting from the recognition position of the first image, and the entire image must be traversed again, which will waste a lot of time and energy. Import the credibility to judge the credibility of the initial recognition result. If the credibility is high, subsequent operations can be performed. If the credibility is low, return to the first step and search and identify again.

[0107] (4) Iterative feedback of classification algorithms. In the traditional classifier testing process, the concept of credibility is not introduced. Before a large number of tests are carried out, the effectiveness of the classifier is unknown. If credibility is introduced as part of the output result, the effectiveness of the classifier can be judged by credibility during the neural network testing process. By using credibility as a parameter and participating in the iterative algorithm process, feedback on the classification effect of the neural network after each iteration can be obtained. This is another form of judgment standard that eliminates covariance error.

[0108] (2) Credibility Modeling of Multi-Output Neural Networks

[0109] 1. Classification based on normalized credibility index

[0110] The multi-output neural network sets N output nodes in the output layer, representing N categories respectively. Each forward propagation output result is an N-dimensional array. The size of each dimension value in the array represents the probability of the corresponding category.

[0111] The probability of each event occurring is between 0 and 1. If we view the classification problem as a probabilistic event, that is, a probabilistic event when a sample belongs to a certain category, and define the value range of each dimension of this N-valued array as [0, 1], the normalized confidence index classification helps the learning algorithm optimize the classification results. A normalized confidence index layer is applied at the end of the neural network to convert the output into a probability distribution.

[0112] Assume that the initial neural network output is y1, y2, ..., yn, and the output is obtained after normalized credibility index classification processing. The credibility model is established based on the output array after normalized credibility index classification. The modeling method is as follows:

[0113] Credibility is determined by the following two components: one is significance, denoted as P r , the other is resolution, denoted as P d , significance P r Is the component representing the classification result as the maximum item in the output vector large enough? The output vector is defined as a vector with each neural network output as a component. In the normalized credibility index output mode, the components of the output vector are not greater than 1 and not less than 0. The significance value is represented by the maximum item value in the output vector, as shown in Formula 1. The resolution P d Whether the component representing the classification result is sufficiently prominent compared with other components in the output vector. The mathematical expression of resolution is expressed as the value of the maximum term divided by the sum of all terms, as shown in Equation 2;

[0114] P r =O m Formula 1

[0115]

[0116]

[0117] Whether the classification result is significant and whether the classification result is discriminative. The occurrence of these two events does not affect each other and is independent of each other. The probability of independent events occurring simultaneously is calculated by multiplying the probabilities of the two events occurring separately. Then, under the classification of the neural network, the probability that the classification result is correct, that is, the calculation formula of the reliability C is shown in Formula 3. Let the output vector be O m Indicates the item with the largest component value in the output vector, 0 t Represents any component in the output vector, and Equation 3 can be used to calculate the credibility of each prediction under the maximum output.

[0118] 2. Intelligent output based on binary

[0119] In order to reduce the number of output results of the artificial neural network, the output categories are numbered in a binary coding manner. When there are many classification categories, the dimension of the output vector is reduced to reduce the computational burden. The dimension of the result vector is the number of bits of the binary number. The value range of the component of the output result vector should be [0, 1]. If the value of a component of the result vector is close to 0, it means that the binary number takes 0 at that bit; if the value of a component of the result vector is close to 1, it takes 1. Then the binary number is converted into a decimal number to obtain the actual classification category.

[0120] Credibility modeling in binary output mode is determined by saliency and resolution, and the output vector is transformed before calculation;

[0121] Suppose there are B outputs after encoding, that is, the output result represents a B-bit binary number, and the output vector is 0 i for The i-th component of a binary digit has two possible results: 0 or 1. The output vector is transformed. The transformed output vector is O′ i for The i-th component of m for The component with the largest median value, 0 i and O′ i The transformation relationship is shown in formula 4, where b i Represents the number on the i-th position of the binary number of this category when the conversion is correct, b i The value is 0 or 1, according to the vector Get the significance P in binary mode r and resolution P d, the results are shown in Equation 5 and Equation 6, and finally the credibility model in binary mode is obtained, as shown in Equation 7:

[0122] O′ i =|O i -(1-b i )| Equation 7

[0123] The single classifier rejection algorithm process of the multi-output neural network with added credibility is as follows Figure 1 As shown, the specific algorithm steps are as follows:

[0124] Step 1: Determine the parameters of the classifier, including the number of inputs, outputs, and intermediate layers;

[0125] Step 2: Test the classifier using a large number of test samples until the requirements are met;

[0126] Step 3: Use the test sample to test, each test sample will get an output vector and a credibility;

[0127] Step 4: Evaluate the classification results according to the credibility C and set the critical value C T , if C≥C T , then adopt the classification result, if C<C T , then reject the classification result.

[0128] (3) Credibility Modeling of Single-Output Neural Networks

[0129] A single-output neural network has only one output node in the output layer, and the output value is between [0, 1]. If it is close to 0, it is judged as yes / no, and if it is close to 1, it is judged as no / yes.

[0130] To establish a credibility model for a single-output neural network, we first use parallel output. Assuming the existence of a multiverse containing multiple world lines, each world line exists independently and equivalently. Parallel output is to establish multiple outputs that are equivalent to but different from the original output to form a set containing multiple outputs. The result of the original output being transformed by a certain function is used as a parallel output. This function is defined as the T function, which must meet the following requirements:

[0131] (1) Reversible, that is, the T function has an inverse function T-1 to perform reverse calculation from parallel output to original output. Let the original output be y and the parallel output be y t , then y t =Z(y),y=T -1 (y t ), T and T -1 They are inverse functions of each other;

[0132] (2) Continuous, that is, T function y t=T(y) in the rectangular coordinate system is a continuous curve without breaks. t The change of changes continuously with the change of the independent variable y;

[0133] (3) Monotonic, that is, in the domain of definition, the T function is monotonic, monotonically increasing or monotonically decreasing. If y1≥y2, then y t1 ≥y t2 (or y t1 ≤y t2 );

[0134] The function that meets the above conditions is called T-function. T-function ensures that there is a close relationship between the parallel output and the original output. Through a series of T-functions, multiple outputs parallel to the original output are quickly established. The credibility modeling of a single output is based on the original output and these parallel outputs.

[0135] T functions include: 1) 2) 3)y t =e y ;

[0136] The modeling method is as follows:

[0137] (1) When establishing the output layer of the neural network, N more components are added, and the expected value of the component is the T transformation of the original expected value. Assume that the set of T functions used is {T 1 , T 2 ,…,T i ,…,T N}, then the expected value of the output vector is

[0138] (2) Test the modified network and get multiple output vectors.

[0139] (3) Perform the corresponding inverse transformation of T transformation on the output components except the first term, and obtain {y′, y 1′ ,y 2′ ,…,y i′ ,…,y N′};

[0140] (4) Solve {y′, y 1′ ,y 2′ ,…,y i′ ,…,y N′}'s variance D;

[0141] (5) Credibility C = 1-D;

[0142] The single classifier rejection algorithm process of the single output neural network with added credibility is as follows Figure 2As shown, the specific algorithm steps are as follows:

[0143] Step 1: Determine the parameters of the classifier, including the number of inputs, outputs, and intermediate layers;

[0144] Step 2: Determine the T function to be used and add parallel output to the network output layer;

[0145] Step 3: Adjust the sample labels to adapt to the network with increased parallel output.

[0146] Step 4: Test the classifier using a large number of test samples until the requirements are met;

[0147] Step 5: Use the test samples to test. Each test sample will get a set of outputs, and a credibility is calculated.

[0148] Step 6: Evaluate the classification results based on the credibility C and set the critical value C T , if C≥C T , then adopt the classification result, if C<C T , then reject the classification result.

[0149] (IV) Credibility-based Intelligent Fusion of Multiple Classifiers

[0150] Based on the credibility calculation model, the credibility of a single maximum output classifier or binary output classifier is calculated. However, in actual operations, to increase the accuracy of the classification results, multiple classifiers are often used to classify the same sample. In this scenario, different classification results will inevitably occur. Due to the different classification accuracy of each classifier, simply averaging all the classification results can sometimes produce worse classification results than using a single highly accurate classifier, which goes against the original intention of using multiple classifiers. If the credibility is used as a weight and the results of all classifiers are weighted and summed, the result will be more accurate than the initial fusion method.

[0151] In the maximum output mode, the output vector Defined as a vector with each neural network output as a component. In binary mode, the output vector refers to the transformed output vector. The output vectors of classifiers with different output modes have different meanings and cannot be fused.

[0152] Formula 8 obtains the fused result vector, where n represents a total of n classifiers. represents the output vector of the i-th classifier, C i is the classification credibility of the i-th classifier in this prediction, Denotes the adjusted output vector obtained after credibility fusion, let O′cj express The jth component of cm express The largest item in the component is the fusion significance P′ r and discrimination P′ d As shown in Equation 9 and Equation 10 respectively, multiply the two to obtain the fused credibility C′, as shown in Equation 11, C′ is the credibility of the fused n classifiers;

[0153]

[0154] P′ r =O′ cm Formula 9

[0155]

[0156]

[0157] When performing multi-classifier recognition on a series of samples, the credibility of the multi-classifier fusion is used to obtain a better recognition result than a single classifier. The credibility fusion algorithm process of the multi-classification is as follows: Figure 3 The specific steps are as follows:

[0158] Step 1: Use n different classifiers to classify the same sample, and obtain n sets of output vectors and n credibility values;

[0159] Step 2: Weight the credibility of these n groups of output vectors according to the established model to obtain a fused output vector;

[0160] Step 3: Get the classification result based on the fused output vector.

[0161] 2. Deep Learning Credibility Fusion

[0162] (1) Handwriting font recognition network architecture

[0163] A convolutional neural network is used to test and learn handwritten fonts, using the maximum output mode for verification. The input of the convolutional neural network is a 28×28 pixel handwritten digit image. Through feature extraction and downsampling, a 10-component output vector is finally obtained.

[0164] The network architecture consists of a data input layer, convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, fully connected layer 1, ReLU layer, fully connected layer 2, and a normalized confidence index layer. The initial input image size is 32×32 pixels, where C represents the convolutional layer, S represents the subsampling layer, and F represents the fully connected layer. The first C1 convolutional layer uses 6 convolution kernels in a single-channel model to obtain 6 feature maps, where the size of each convolution kernel is 5×5. The initial input image is convolved with the convolution kernel, and the resulting feature map size is (32-5+1)×(32-5+1)=28×28. The number of parameters used in this process is (5×5+1)×6=156, and the number of connections is (5×5+1)×28×28×6=122304, where 5×5 is the convolution template parameter, 1 is the bias parameter, and 28×28 is the size of the image after convolution.

[0165] Subsampling layer 1-S2 uses 6 feature maps, each of which is 14×14 in size. The hidden unit of each feature map is connected to the 2×2 unit of the feature map corresponding to the previous layer C1. The calculation process is: the values ​​in the 2×2 unit are added and then multiplied by the weight w, plus a bias parameter b. Each feature map shares the weight, and then uses the activation function to process it, and the obtained value is used as the value of the corresponding unit. The side length of each feature map in the S2 layer is half of that of the previous layer C1. The S2 layer requires 2×6=12 parameters and the number of connections is (4+1)×14×14×6=5880.

[0166] Convolutional layer 2-C3 uses 14 channels and 16 convolution kernels. The size of the convolution kernel is 5×5. C3 is partially connected to the previous layer S2. C3 includes 16 feature maps. The size of each feature map is (14-5+1)×(14-5+1)=10×10. Each feature map is only connected to part of the feature maps in the previous layer S2. The calculation process is described with the 0th feature map: 1 convolution kernel is used to perform convolution calculations with the 3 feature maps of the S2 layer respectively, and then the convolution results are added, plus a bias, and then processed by the activation function to obtain the corresponding feature map. The number of parameters required is (5×5×3+1)×6+(5×5×4+1)×9+5×5×6+1=1516, where 5×5 is the convolution parameter. The convolution kernel has 3, 4, and 6 convolution templates respectively, and the number of connections is 1516×10×10=151600.

[0167] Subsampling layers 2-S4 set 16 feature maps, each feature map has a size of 5×5, the number of parameters is 16×2=32, and the number of connections is (4+1)×5×5×16=2000;

[0168] Convolutional layer 3-C5 sets 120 convolution kernels, the size of the convolution kernel is 5×5, including 120 feature maps, the convolution kernel has 16 convolution templates, the size of the feature map is 1×1, just becoming fully connected, the C5 layer has 120×(5×5×16+1)=48120 parameters;

[0169] The fully connected layer F6 has 86 neurons, each of which is fully connected to C5. The number of connections and parameters are both 86×120=10164. The F6 layer obtains an 86-dimensional feature, which is subsequently used for classification prediction.

[0170] (2) Face Gender Recognition Network Architecture

[0171] The network input data uses 3 color channels, the initial image size is 816×816, and the resized image is 256×256. The resized image is then randomly cropped to a size of 224×224 and used as the input of the network.

[0172] Cascaded 3×3 convolution kernels are used as the underlying feature extraction, and edge expansion is adopted. The receptive field of three cascaded 3×3 convolution kernels is 7×7, the weight parameters of the 7×7 convolution kernel are 7×7×64=49×64, and the weight parameters of the cascaded small-size convolution kernel are 3×3×3×64=27×64. The fusion of middle-level features and high-level features is achieved by crossing the convolution layer. The L9 and L10 layers perform complex calculations in each local receptive field to extract various potential nonlinear features. Lll_a uses global average pooling, and the L7 layer spans L8 to 10 layers and is directly fully connected to the last hidden layer. The BN layer is added to play a normalization role.

[0173] (III) Single classifier credibility rejection of multi-output classifier

[0174] The credibility modeling of the multi-output neural network is carried out. The handwritten character recognition network model has multiple outputs, and the handwritten digit recognition output components have 10, representing the 10 digits 0-9.

[0175] Comparing traditional neural networks with neural networks that consider credibility, to examine whether the latter can eliminate errors and improve accuracy, a detection model was established. The steps are as follows:

[0176] Step a: Establish multiple classifiers by adjusting the number of iterations. In the maximum output mode, the iterations are 10000, 9000, 8000, 7000, 6000, 5000, 4000, 3000, 2000, and 1000 respectively. The obtained classifiers are numbered as classifier 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10.

[0177] Step b: Test the above ten classifiers using the same test samples, calculate the credibility of the classification results, and calculate and analyze the different distributions of the credibility values ​​in the cases of correct and incorrect classification. Figure 4 shown.

[0178] Step c: Set the credibility threshold for the classifier, and analyze the changes in the recognition accuracy when setting different thresholds. The recognition error rate and changes of each classifier are analyzed when there is no threshold and when the threshold is set to 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, and 0.50. The recognition error rate is the total test probability of the test sample not matching the label in the classification result of the classifier after removing the examples with credibility lower than the threshold. Figure 4 It turns out that:

[0179] (1) When the same classifier is used for classification, the average confidence level of the correct classification result is higher than the average confidence level of the incorrect classification result. The confidence level measures the reliability of the classification result. When the confidence level of a prediction is very high, it is recommended to accept the recognition result. When the confidence level of a prediction is very low, it is not recommended to accept the recognition result.

[0180] (2) However, the overall credibility of the classifier is independent of the accuracy of the classifier. When the average credibility is high, the accuracy of the classifier is not necessarily high; when the average credibility is low, the accuracy of the classifier is not necessarily low. Credibility and recognition rate are different concepts. Credibility and recognition rate evaluate different objects. Credibility is for a classification task, not for the classifier, while recognition rate evaluates the entire classifier.

[0181] (3) The credibility distribution of each classifier is different. We cannot simply use the average credibility value of multiple classifications to measure the accuracy of a classifier, nor can we use the average credibility of different classifiers to judge the pros and cons of each classifier.

[0182] 1) After introducing credibility into the neural network and eliminating some unreliable results based on the credibility threshold, the recognition error rate of the classifier generally decreases. When the credibility threshold is above 0.5, the error rate reaches 0. The introduction of credibility optimizes the recognition process of the classifier, avoids receiving incorrect classification results, and achieves more accurate classification effects.

[0183] 2) When each classifier has a very low rejection rate (the ratio of test samples that the system rejects due to low confidence levels to the total number of test samples), the recognition accuracy is improved. The vast majority of items rejected by the system are incorrect in the recognition results. There are not many cases where the confidence values ​​are very low but the recognition results are actually correct. The confidence value plays a role in evaluating the reliability of the recognition results.

[0184] 3) For different classifiers, different optimization effects are achieved by using different credibility thresholds. It is necessary to strike a balance between the recognition accuracy and the rejection rate. Different classifiers are suitable for different credibility thresholds, and their values ​​are related to the average credibility when the classification is wrong.

[0185] (IV) Single-classifier credibility rejection of single-output classifier

[0186] The credibility modeling effect of the single-class output network was verified by face gender recognition. The network output components were changed from two to one. The judgment standard for the single output is: if the result is close to 0, it is judged as male, and if the result is close to 1, it is judged as female. This does not conflict with the original design purpose of the network and can meet the network requirement of a single output. Therefore, this method is used to modify the network. The specific steps are as follows:

[0187] Step I: Different classifiers are obtained by adjusting the number of iterations. The same test set is tested. Five classifiers are obtained by iterating 10,000, 15,000, 20,000, 25,000, and 30,000 times, respectively. They are numbered as classifiers A, B, C, D, and E.

[0188] Step II: For the five classifiers obtained above, three parallel outputs are added to the original network single output and modeled. The T functions used are y t =y 2 ;y t =e y ;

[0189] Step III: Use the same test samples to test classifiers A, B, C, D, and E, record their respective accuracy rates and calculated credibility values, and calculate and analyze the different distributions of credibility values ​​in correct and incorrect classification cases.

[0190] The results are as follows Figure 5Compared with multi-output classifiers, the credibility modeling method for single-output classifiers is different. However, after establishing the credibility model, similar results were obtained: for the same classifier, the average credibility when the classification is correct is higher than when the classification is incorrect. It is feasible to use credibility to measure the reliability of a prediction result. At the same time, the lack of correlation between average credibility and recognition accuracy also proves that credibility is a method to help optimize the algorithm and improve the prediction effect from another perspective, further illustrating that average credibility cannot be used to measure the accuracy of a classifier.

[0191] (V) Credibility Fusion of Multi-classifiers with Multi-output Classifiers

[0192] For multi-output classifiers, the dataset and network architecture used are the same as those for handwritten character recognition. In order to examine whether the classifier after credibility fusion has improved the recognition accuracy compared with the original classifiers, the classifier output mode is the maximum output. The steps are as follows:

[0193] Step A: Change the initial weights of the neural network and iterate 1000 times to obtain 5 classifiers, numbered I, II, III, IV, and V. Use the same test samples to test these 5 classifiers and obtain the recognition accuracy of each classifier;

[0194] Step B: Use the model established above to fuse the above five classifiers, numbered V, and test the same test samples to obtain the recognition accuracy after fusion;

[0195] Step C: Change the number of iterations and repeat the operations of steps 1 and 2 to obtain groups with the number of iterations being 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, and 10000;

[0196] Step 4: Compare the recognition accuracy of the five classifiers in the same group with the recognition accuracy of the fusion classifier;

[0197] The experimental results are as follows Figure 6 ,Depend on Figure 6 It can be seen that:

[0198] (1) In all ten experimental groups, the recognition accuracy after credibility fusion was greater than or equal to the recognition accuracy of the five original classifiers. Compared with the classifiers with higher recognition accuracy, credibility fusion did not reduce their accuracy, but instead improved their recognition accuracy to varying degrees. In particular, compared with the classifiers with lower recognition accuracy, the accuracy was significantly improved after credibility fusion.

[0199] (2) The credibility fusion of multiple classifiers not only ensures that the recognition accuracy after fusion is not negatively affected by the low recognition rate classifier and remains at the highest level of recognition accuracy in the original classifier, but also can slightly improve the recognition accuracy at this level, thereby achieving the effect of optimizing the recognition effect and improving the recognition accuracy. It is a feasible and effective multi-classifier fusion method.

[0200] 3. Experimental Analysis

[0201] A single classifier credibility rejection experiment of a multi-output network model for handwritten character recognition, a single classifier credibility rejection experiment of a single output network model for face gender recognition, and a multi-classifier credibility fusion experiment of a multi-output network model taking handwritten character recognition as an example.

[0202] The experimental results are summarized as follows:

[0203] (1) Although the credibility modeling methods of the multi-output network model and the single-output network model are different, the results of the two single-classifier rejection experiments show that credibility is useful for measuring a certain prediction result. For the results of misclassification, the credibility is very low; for the results of correct classification, the credibility is very high. Therefore, by setting the credibility threshold to filter the classification results, it is possible to effectively eliminate the cases of misidentification.

[0204] (2) There is no relationship between recognition accuracy and average credibility. A high recognition accuracy does not necessarily mean a high average credibility. This verifies the statement that credibility and recognition accuracy are two different measures. Recognition accuracy cannot be used to measure the credibility of each prediction. Similarly, average credibility is not suitable for measuring the overall recognition effect of a classifier.

[0205] (3) For different classifiers, the distribution of credibility values ​​is different. The credibility threshold should be set according to the actual situation of the classifier to achieve the best rejection effect. Comprehensive comparison shows that setting the credibility threshold to 0.3 is a relatively good choice.

[0206] (4) Fusing the credibility of the classification results of multiple classifiers can improve the final recognition accuracy. The recognition rate after fusion is higher than that of all the original classifiers. This multi-classifier credibility fusion method is not affected by the shortcomings of the classifier with low recognition rate, but maximizes the advantages of each classifier, achieving the effect of one plus one being greater than two. Moreover, the calculation of credibility fusion is just simple addition, subtraction, multiplication and division, which achieves good results at a very low computational cost.

Claims

1. A neural network credibility modeling method for intelligent processing of big data, characterized by: Starting with evaluating a single prediction result of a classifier, we use the output layer of the network model itself to construct a credibility model for the deep neural network to measure the trustworthiness of a single prediction. Based on convolutional neural networks, we conduct credibility modeling and effectiveness testing for classifiers with different output types, using handwritten character recognition as a multi-output neural network and facial gender recognition as a single-output neural network. Furthermore, we use credibility to fuse multiple classifiers to obtain more reliable results. 1) Credibility Modeling of Big Data Neural Networks: This includes credibility modeling of multi-output neural networks, credibility modeling of single-output neural networks, and intelligent fusion of multiple classifiers. Credibility modeling of multi-output neural networks is divided into classification based on normalized credibility index and intelligent output based on binary. Credibility modeling of big data neural networks establishes mathematical models of the credibility of deep neural networks in two output modes: multi-output classifiers and single-output classifiers. In addition, the credibility fusion of multiple classifiers is modeled to construct a deep neural network classification algorithm that considers credibility. 2) Deep Learning Credibility Fusion: This includes handwriting recognition network architecture, facial gender recognition network architecture, single-classifier credibility rejection of multi-output classifiers, single-classifier credibility rejection of single-output classifiers, and multi-classifier credibility fusion of multi-output classifiers. Through credibility-based handwriting recognition, facial gender recognition, and credibility fusion methods, a credibility model for evaluating the effectiveness of a prediction result is established. The credibility fusion of multiple classifiers is superior to all original classifiers.

2. The big data intelligent processing neural network credibility modeling method according to claim 1 is characterized in that: Classification based on normalized credibility index: The multi-output neural network sets N output nodes in the output layer, representing N categories respectively. Each forward propagation output result is an N-dimensional array. The size of each dimension in the array represents the probability of the corresponding category. The probability of each event occurring is between 0 and 1. The classification problem is considered a probabilistic event, where a sample belongs to a certain category. The range of each dimension of the N-th array is set to [0, 1]. Normalized confidence index classification helps the learning algorithm optimize the classification results. A normalized confidence index layer is added at the end of the neural network to convert the output into a probability distribution. Assume that the initial neural network output is y1, y2, ..., yn, and the output is obtained after normalized credibility index classification processing. The credibility model is established based on the output array after normalized credibility index classification. The modeling method is as follows: Credibility is determined by the following two components: one is significance, denoted as P r , the other is resolution, denoted as P d , significance P r Is the component representing the classification result as the maximum item in the output vector large enough? The output vector is defined as a vector with each neural network output as a component. In the normalized credibility index output mode, the components of the output vector are not greater than 1 and not less than 0. The significance value is represented by the maximum item value in the output vector, as shown in Formula 1. The resolution P d Whether the component representing the classification result is sufficiently prominent compared with other components in the output vector. The mathematical expression of resolution is expressed as the value of the maximum term divided by the sum of all terms, as shown in Equation 2; P r =O m Formula 1 Whether the classification result is significant and whether the classification result is discriminative. The occurrence of these two events does not affect each other and is independent of each other. The probability of independent events occurring simultaneously is calculated by multiplying the probabilities of the two events occurring separately. Then, under the classification of the neural network, the probability that the classification result is correct, that is, the calculation formula of the reliability C is shown in Formula 3. Let the output vector be O m Indicates the item with the largest component value in the output vector, 0 t Represents any component in the output vector, and Equation 3 can be used to calculate the credibility of each prediction under the maximum output.

3. The big data intelligent processing neural network credibility modeling method according to claim 1 is characterized in that: Intelligent output based on binary: binary coding is used to number the output categories. When there are many classification categories, the dimension of the output vector is reduced. The dimension of the result vector is the number of bits of the binary number. The component value range of the output result vector should be [0, 1]. If the value of a component of the result vector is close to 0, it means that the binary number takes 0 at that bit; if the value of a component of the result vector is close to 1, it takes 1. Then the binary number is converted into a decimal number to obtain the actual classification category; Credibility modeling in binary output mode is determined by saliency and resolution, and the output vector is transformed before calculation; Suppose there are B outputs after encoding, that is, the output result represents a B-bit binary number, and the output vector is 0 i for The i-th component of a binary digit has two possible results: 0 or 1. The output vector is transformed. The transformed output vector is O′ i for The i-th component of m for The component with the largest median value, 0 i and O′ i The transformation relationship is shown in formula 4, where b i Represents the number on the i-th position of the binary number of this category when the conversion is correct, b i The value is 0 or 1, according to the vector Get the significance P in binary mode r and resolution P d , the results are shown in Equation 5 and Equation 6, and finally the credibility model in binary mode is obtained, as shown in Equation 7: O' i =|O i -(1-b i )| Expression 7 The steps of the single classifier rejection algorithm of the multi-output neural network with added credibility are as follows: Step 1: Determine the parameters of the classifier, including the number of inputs, outputs, and intermediate layers; Step 2: Test the classifier using a large number of test samples until the requirements are met; Step 3: Use the test sample to test, each test sample will get an output vector and a credibility; Step 4: Evaluate the classification results according to the credibility C and set the critical value C T , if C≥C T , then adopt the classification result, if C<C T , then reject the classification result.

4. The method for intelligent processing of big data by neural network credibility modeling according to claim 1 is characterized in that: Credibility modeling of single-output neural network: A single-output neural network has only one output node in the output layer, and the output value is between [0, 1]. A value close to 0 is judged as yes / no, and a value close to 1 is judged as no / yes. To establish a credibility model for a single-output neural network, we first use parallel output. Assuming the existence of a multiverse containing multiple world lines, each world line exists independently and equivalently. Parallel output is to establish multiple outputs that are equivalent to but different from the original output to form a set containing multiple outputs. The result of the original output being transformed by a certain function is used as a parallel output. This function is defined as the T function, which must meet the following requirements: (1) Reversible, that is, the T function has an inverse function T-1 to perform reverse calculation from parallel output to original output. Let the original output be y and the parallel output be y t , then y t =Z(y),y=T -1 (y t ), T and T -1 They are inverse functions of each other; (2) Continuous, that is, T function y t =T(y) in the rectangular coordinate system is a continuous curve without breaks. t The change of changes continuously with the change of the independent variable y; (3) Monotonic, that is, in the domain of definition, the T function is monotonic, monotonically increasing or monotonically decreasing. If y1≥y2, then y t1 ≥y t2 (or y t1 ≤y t2 ); The function that meets the above conditions is called T-function. T-function ensures that there is a close relationship between the parallel output and the original output. Through a series of T-functions, multiple outputs parallel to the original output are quickly established. The credibility modeling of a single output is based on the original output and these parallel outputs. T functions include: 1) 2)y t =y 2 ;3)y t =e y ; The modeling method is as follows: (1) When establishing the output layer of the neural network, N more components are added, and the expected value of the component is the T transformation of the original expected value. Assume that the set of T functions used is {T 1 , T 2 ,…,T i ,…,T N }, then the expected value of the output vector is (2) Test the modified network and get multiple output vectors. (3) Perform the corresponding inverse transformation of T transformation on the output components except the first term, and obtain {y', y 1 ',y 2 ',…,y i ',…,y N '}; (4) Solve {y', y 1 ',y 2 ',…,y i ',…,y N '}'s variance D; (5) Credibility C = 1-D; The steps of the single classifier rejection algorithm of the single-output neural network with added credibility are as follows: Step 1: Determine the parameters of the classifier, including the number of inputs, outputs, and intermediate layers; Step 2: Determine the T function to be used and add parallel output to the network output layer; Step 3: Adjust the sample labels to adapt to the network with increased parallel output. Step 4: Test the classifier using a large number of test samples until the requirements are met; Step 5: Use the test samples to test. Each test sample will get a set of outputs, and a credibility is calculated. Step 6: Evaluate the classification results based on the credibility C and set the critical value C T , if C≥C T , then adopt the classification result, if C<C T , then reject the classification result.

5. The big data intelligent processing neural network credibility modeling method according to claim 1 is characterized in that: Credibility intelligent fusion of multiple classifiers: In the maximum output mode, the output vector Defined as a vector with each neural network output as a component. In binary mode, the output vector refers to the transformed output vector. The output vectors of classifiers with different output modes have different meanings and cannot be fused. Formula 8 obtains the fused result vector, where n represents a total of n classifiers. represents the output vector of the i-th classifier, C i is the classification credibility of the i-th classifier in this prediction, Denotes the adjusted output vector obtained after credibility fusion, let O′ cj express The jth component of cm express The largest item in the component is the fusion significance P′ r and discrimination P′ d As shown in Equation 9 and Equation 10 respectively, multiply the two to obtain the fused credibility C′, as shown in Equation 11, C′ is the credibility of the fused n classifiers; P′ r =O′ cm Formula 9 When performing multi-classifier recognition on a series of samples, the multi-classifier fusion using credibility can obtain recognition results that are better than those of a single classifier. The process steps of the multi-classification credibility fusion algorithm are as follows: Step 1: Use n different classifiers to classify the same sample, and obtain n sets of output vectors and n credibility values; Step 2: Weight the credibility of these n groups of output vectors according to the established model to obtain a fused output vector; Step 3: Get the classification result based on the fused output vector.

6. The method for intelligent processing of big data by neural network credibility modeling according to claim 1 is characterized in that: Handwriting recognition network architecture: A convolutional neural network is used to test and learn handwriting, using a maximum output mode for verification. The convolutional neural network takes a 28×28 pixel handwritten digit image as input and generates a 10-component output vector through feature extraction and downsampling. The network architecture consists of a data input layer, convolution layer 1, pooling layer 1, convolution layer 2, pooling layer 2, fully connected layer 1, ReLU layer, fully connected layer 2, and a normalized confidence index layer. The initial input image size is 32×32 pixels, where C represents the convolution layer, S represents the subsampling layer, and F represents the fully connected layer. The first C1 convolution layer uses 6 convolution kernels in a single-channel model to obtain 6 feature maps, where the size of each convolution kernel is 5×5. The initial input image is convolved with the convolution kernel, and the resulting feature map size is (32-5+1)×(32-5+1)=28×28. The number of parameters used in this process is (5×5+1)×6=156, and the number of connections is (5×5+1)×28×28×6=122304, where 5×5 is the convolution template parameter, 1 is the bias parameter, and 28×28 is the size of the image after convolution. Subsampling layer 1-S2 uses 6 feature maps, each of which is 14×14 in size. The hidden unit of each feature map is connected to the 2×2 unit of the feature map corresponding to the previous layer C1. The calculation process is: the values ​​in the 2×2 unit are added and then multiplied by the weight w, plus a bias parameter b. Each feature map shares the weight, and then uses the activation function to process it, and the obtained value is used as the value of the corresponding unit. The side length of each feature map in the S2 layer is half of that of the previous layer C1. The S2 layer requires 2×6=12 parameters and the number of connections is (4+1)×14×14×6=5880. Convolutional layer 2-C3 uses 14 channels and 16 convolution kernels. The size of the convolution kernel is 5×5. C3 is partially connected to the previous layer S2. C3 includes 16 feature maps. The size of each feature map is (14-5+1)×(14-5+1)=10×10. Each feature map is only connected to part of the feature maps in the previous layer S2. The calculation process is described with the 0th feature map: 1 convolution kernel is used to perform convolution calculations with the 3 feature maps of the S2 layer respectively, and then the convolution results are added, plus a bias, and then processed by the activation function to obtain the corresponding feature map. The number of parameters required is (5×5×3+1)×6+(5×5×4+1)×9+5×5×6+1=1516, where 5×5 is the convolution parameter. The convolution kernel has 3, 4, and 6 convolution templates respectively, and the number of connections is 1516×10×10=151600. Subsampling layers 2-S4 set 16 feature maps, each feature map has a size of 5×5, the number of parameters is 16×2=32, and the number of connections is (4+1)×5×5×16=2000; Convolutional layer 3-C5 sets 120 convolution kernels, the size of the convolution kernel is 5×5, including 120 feature maps, the convolution kernel has 16 convolution templates, the size of the feature map is 1×1, just becoming fully connected, the C5 layer has 120×(5×5×16+1)=48120 parameters; The fully connected layer F6 has 86 neurons, each of which is fully connected to C5. The number of connections and parameters are both 86×120=10164. The F6 layer obtains an 86-dimensional feature, which is subsequently used for classification prediction.

7. The method for intelligently processing big data and neural network credibility modeling according to claim 1 is characterized in that: Face gender recognition network architecture: The network input data uses three color channels, the initial image size is 816×816, and the resized image is resized to 256×256. The resized image is then randomly cropped to a size of 224×224 and used as the network input; Cascaded 3×3 convolution kernels are used as the underlying feature extraction, and edge expansion is adopted. The receptive field of three cascaded 3×3 convolution kernels is 7×7, the weight parameters of the 7×7 convolution kernel are 7×7×64=49×64, and the weight parameters of the cascaded small-size convolution kernel are 3×3×3×64=27×64. The fusion of middle-level features and high-level features is achieved by crossing the convolution layer. The L9 and L10 layers perform complex calculations in each local receptive field to extract various potential nonlinear features. Lll_a uses global average pooling, and the L7 layer spans L8 to 10 layers and is directly fully connected to the last hidden layer. The BN layer is added to play a normalization role.

8. The method for intelligent processing of big data by neural network credibility modeling according to claim 1 is characterized in that: Credibility rejection of a single classifier for a multi-output classifier: Credibility modeling is performed on a multi-output neural network. The handwritten character recognition network model has multiple outputs, and the handwritten digit recognition output components have 10, representing the 10 digits 0-9. To build a detection model, the steps are as follows: Step a: Establish multiple classifiers by adjusting the number of iterations. In the maximum output mode, the iterations are 10000, 9000, 8000, 7000, 6000, 5000, 4000, 3000, 2000, and 1000 respectively. The obtained classifiers are numbered as classifier 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. Step b: Test the above ten classifiers using the same test samples, calculate the credibility of the classification results, and calculate and analyze the different distributions of the credibility values ​​in the cases of correct and incorrect classification; Step c: Set the credibility threshold for the classifier, and analyze the changes in the recognition accuracy when setting different critical values. The recognition error rate and changes of each classifier are analyzed when there is no critical value and when the critical value is set to 0.05, 0.10, 0.15, 0.20, 0.25, 0.30, 0.35, 0.40, 0.45, and 0.

50. The recognition error rate is the total test probability that the classification result of the test sample in the classifier does not match the label after removing the examples with credibility lower than the critical value.

9. The method for intelligent processing of big data by neural network credibility modeling according to claim 1, characterized in that: Single-output classifier credibility rejection: Face gender recognition is used to verify the credibility modeling effect of the single-output network. The network output components are changed from two to one. The judgment standard for the single output is: if the result is close to 0, it is judged as male, and if the result is close to 1, it is judged as female. This does not conflict with the original design purpose of the network and can meet the network requirement of a single output. Therefore, this method is used to modify the network. The specific steps are as follows: Step I: Different classifiers are obtained by adjusting the number of iterations. The same test set is tested. Five classifiers are obtained by iterating 10,000, 15,000, 20,000, 25,000, and 30,000 times, respectively. They are numbered as classifiers A, B, C, D, and E. Step II: For the five classifiers obtained above, three parallel outputs are added to the original network single output and modeled. The T functions used are y t =y 2 ;y t =e y ; Step III: Use the same test samples to test classifiers A, B, C, D, and E, record their respective accuracy rates and calculated credibility values, and calculate and analyze the different distributions of credibility values ​​in correct and incorrect classification cases.

10. The method for intelligent processing of big data by neural network credibility modeling according to claim 1, characterized in that: Multi-classifier credibility fusion of multi-output classifiers: The classifier output mode is the maximum output, and the steps are as follows: Step A: Change the initial weights of the neural network and iterate 1000 times to obtain 5 classifiers, numbered I, II, III, IV, and V. Use the same test samples to test these 5 classifiers and obtain the recognition accuracy of each classifier; Step B: Use the model established above to fuse the above five classifiers, numbered V, and test the same test samples to obtain the recognition accuracy after fusion; Step C: Change the number of iterations and repeat the operations of steps 1 and 2 to obtain groups with the number of iterations being 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, and 10000; Step 4: Compare the recognition accuracy of the five classifiers in the same group with the recognition accuracy of the fusion classifier.

Images