Data Stream Driven Dynamic Node Width Learning Image Classification Method

Abstract

This invention discloses a data flow-driven dynamic node width learning image classification method, comprising the following steps: S1, setting the initial number of mapped feature nodes and augmentation nodes, inputting initial batch image data, and establishing a width learning model based on the set parameters; S2, determining whether a new image data stream has arrived or whether the preset termination condition has not been met. If not, outputting the trained width learning model and testing its performance; if yes, proceeding to step S3; S3, calculating the number of augmentation nodes to be added; S4, adjusting the node configuration of the width learning model, updating the weights of the width learning model using incremental data streams, and jumping back to step S2. This invention automatically calculates the adjustment range of augmentation nodes to adapt to data flow scenarios through data-driven methods, solving the problem of poor performance caused by a large influx of data in data flow scenarios, and also avoiding the tedious steps of manually setting model parameters.

Description

Technical Field

[0001] This invention belongs to the field of width learning technology, specifically relating to a data flow-driven dynamic node width learning image classification method. Background Technology

[0002] Image data stream classification has attracted much attention as an important task in data mining. However, the dynamic nature of data streams presents new challenges and demands on traditional classification models in terms of computation time, rapid model updates, and classification accuracy.

[0003] In recent years, research on data stream classification has garnered widespread attention, leading to the development of numerous related data stream classification algorithms. Among these, deep learning methods, due to their superior performance, have become the mainstream approach, used to solve various practical problems. However, the large number of parameters and training methods in deep structures increase the model's time and resource consumption. Width learning systems are effective single-hidden-layer incremental models that can build and update high-resolution models in a short time. These characteristics make them more suitable for data stream scenarios and simultaneously address the time consumption issues faced by deep learning methods.

[0004] Width learning systems can improve model performance to adapt to the ever-increasing volume of data in streaming scenarios by enhancing node adjustments. However, standard width learning systems use manual or static node settings, which lack flexibility, and improper settings can lead to additional time and resource consumption. Setting the number of nodes too small will affect model optimization efficiency, while setting it too large will result in lost model parameters and excessive resource consumption. In particular, in streaming scenarios, image data acquisition is often multi-batch and unequal in quantity, posing a significant challenge to network models using manual or static node settings. Furthermore, when large amounts of data flood in, static settings cannot quickly achieve high performance and may even have negative effects. Therefore, designing adaptive dynamic node adjustment and model update strategies for streaming scenarios is crucial for the widespread application of width learning in image classification. Summary of the Invention

[0005] The main objective of this invention is to overcome the shortcomings and deficiencies of the prior art and propose a data flow-driven dynamic node width learning image classification method.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A dataflow-driven, dynamic node-width learning image classification method includes the following steps:

[0008] S1. Set the number of initial mapping feature nodes and enhancement nodes, input the initial batch image data, and build a width learning model based on the set parameters;

[0009] S2. Determine whether a new image data stream has arrived or whether the preset termination condition has not been met. If not, output the trained width learning model and test its performance; if yes, proceed to step S3.

[0010] S3. Calculate the number of enhancement nodes that need to be added;

[0011] S4. Adjust the node configuration of the width learning model, update the weights of the width learning model using incremental data stream, and jump to step S2.

[0012] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0013] 1. Compared with current deep learning methods, the method of this invention uses incremental update width learning to solve the problem of data flow scenarios, which can effectively solve the problems of time and resource consumption. Secondly, by adaptively calculating the number of augmenting nodes in a data-driven manner in the flow scenario, it can solve the problem of poor model performance caused by the addition of large amounts of data and improve the stability of the model. On the other hand, this mechanism avoids the tedious steps of manually setting model parameters and the increase in training time caused by setting inappropriate model parameters. It has the advantages of high recognition accuracy and simple algorithm. Finally, the two update strategies provided can solve the problems of concept drift and rapid update in data flow respectively. Attached Figure Description

[0014] Figure 1 This is a flowchart of the method of the present invention;

[0015] Figure 2 This is a flowchart of the training process for the width learning model in this invention;

[0016] Figure 3 This is a schematic diagram of MNIST data in the embodiment;

[0017] Figure 4 This is a schematic diagram of the result using an initial node of 100 in the embodiment;

[0018] Figure 5 This is a schematic diagram of the result using an initial node of 200 in the embodiment;

[0019] Figure 6 This is a schematic diagram of the result using an initial node of 500 in the embodiment. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0021] Example

[0022] like Figure 1As shown, the present invention provides a data flow-driven dynamic node width learning image classification method, which includes the following steps:

[0023] S1. Set the number of initial mapping feature nodes and enhancement nodes, input the initial batch image data, and build a width learning model based on the set parameters;

[0024] Among them, such as Figure 2 As shown, the specific steps for building a width learning model based on the set parameters are as follows:

[0025] Using the initial image input X, calculate the feature map node Z. n ≡[Z1, Z2, ..., Z n The process is a linear mapping, where the i-th group of mapping features Z i The calculation is as shown in formula (1), and the connection weight W ei and β ei It is generated randomly. For linear mappings, each set of features contains k mapping features;

[0026]

[0027] By mapping node Z n Calculate the enhancement node H mm ≡[H1, H2, ..., H m This process is a non-linear mapping, where the enhanced feature H... j The calculation is as shown in formula (2), and the connection weight W hj and β hj It is randomly generated, ζ(Z, W) hj ,β hj () is a non-linear activation function;

[0028] H j ≡ζ(Z n W hj +β hj ), j = 1, 2, ..., m (2)

[0029] The mapping node represents the linear characteristics of the data, and the enhancement node represents the nonlinear characteristics of the data;

[0030] Linear and nonlinear features are combined into a feature layer A = [Z|H], which is then connected to the model output layer Y.

[0031] Since the information of the model's output layer Y is known, i.e., the category corresponding to the data, we only need to calculate the connection weights W between the feature layer and the output layer. The calculation formula is as follows:

[0032] W = A -1 Y (3)

[0033] In the calculation, ridge regression is used to solve for the weight matrix, that is, W is solved by the optimization problem of formula (3):

[0034]

[0035] Where σ1=σ2=v=u=2, we can solve for:

[0036] W=(λI+AA T ) -1 A T Y(5).

[0037] S2. Determine whether a new image data stream has arrived or whether the preset termination condition has not been met. If not, output the trained width learning model and test its performance; if yes, proceed to step S3.

[0038] The preset termination condition is that the test accuracy of the width learning model reaches the preset test accuracy.

[0039] S3. Calculate the number of enhancement nodes that need to be added. There are two methods for calculation, as follows:

[0040] The number of augmentation nodes in the model is adjusted based on the training sample data currently used for width learning and the number of samples in the new data stream. The new number of nodes m′(t+1) is calculated by formula (6):

[0041]

[0042] Where m(t) is the number of augmentation nodes at the current time, im(t) is the number of samples in the new data stream, and l(t) is the number of samples in the training data used by the model at the current time. It's a floor operation; the number of enhancement nodes needed, Δm, is...

[0043] Based on the current number of augmentation nodes m(t) in the width learning model, the number of augmentation nodes in the model is adjusted, and the new number of nodes m′(t+1) is calculated by formula (7):

[0044]

[0045] The number of enhancement nodes that need to be added, Δm, is...

[0046] S4. Adjust the node configuration of the width learning model, update the weights of the width learning model using incremental data stream, and jump to step S2.

[0047] Specifically, updating the weights of the width learning model using incremental data streams includes retraining and incremental learning methods.

[0048] The retraining method is as follows:

[0049] The previously used training image data X(t) is merged with the newly arrived incremental data stream B(t+1) to form new training data X(t+1) = [X(t)|(t+1)];

[0050] Based on the new training data X(t+1), calculate the feature mapping node Z. n (t+1);

[0051] Calculate the enhanced feature H based on the calculated mapped feature nodes and the adjusted number of enhanced nodes m′(t+1). m (t+1), merge the mapped features and enhanced features into a feature layer A(t+1) = [Z|H], and connect it to the output layer Y of the width learning model;

[0052] The output layer connection weights are calculated using formulas (3) and (4).

[0053] The incremental learning method is as follows:

[0054] Based on the newly arrived incremental data stream B(t+1), calculate the corresponding feature A according to the current model parameter settings. x As shown in formula (8):

[0055]

[0056] in, It is the mapping node corresponding to the incremental data; W ei β ei W hj and β ej The weights of the model in the t-th batch of data;

[0057] The corresponding new features at this time Calculated using formula (9):

[0058]

[0059] At the same time, it is also necessary to consider the current mapping characteristics. Calculate the enhancement feature H corresponding to adding Δm enhancement nodes. Δm ≡[H1, H2, ..., H Δm ], where the j-th enhancement feature H j Calculate using formula (10):

[0060]

[0061] The features after adding data and enhancing nodes are calculated as shown in formula (11):

[0062]

[0063] The model weights W(t) obtained based on the previous training data X(t), and the connection weights W between the features generated by the incremental samples and the output layer. x (t+1) can be calculated directly using formula (5);

[0064] The new weights W(t+1) = [W(t)|W x [(t+1)]; After adding the enhancement node, the final weights of the model are updated using formula (12):

[0065]

[0066]

[0067]

[0068]

[0069]

[0070] Among them, Y x These are the labels for the incremental data B(t+1).

[0071] To further illustrate the technical effectiveness of the method of this invention, the MNIST handwritten digit dataset was used for testing. A total of 70,000 data points were included, with 60,000 used as the training set and 10,000 as the test set. Each grayscale image has 28×28 pixels and covers 10 categories, representing the digits 0 to 9. The dataset is shown below. Figure 3 As shown, the training dataset is divided into 11 unequal batches to form a data stream scenario (1 initial batch and 10 incremental batches). In particular, the data stream contains large amounts of data.

[0072] BLS with a data-driven dynamic node mechanism was compared with traditional and statically configured BLS. The initial number of mapped features in all three experiments was set to 120 (10×12), and the initial number of augmentation nodes was set to 100, 200, and 500, respectively. Traditional BLS did not adjust the model structure, statically configured BLS augmentation nodes were added statically, while the other BLS augmentation node method used the data-driven approach corresponding to formula (6). The model update strategy for all three was retraining. The results are as follows: Figure 4 , Figure 5 as well as Figure 6 As shown.

[0073] The analysis of the above results shows that the method of this invention is more effective than the standard breadth-of-range learning method in MNIST. It can quickly find a better model structure and improve the final performance of the model, saving computational costs and human resources while improving efficiency. Even if the initial performance is lower than that of the standard BLS and static adjustment mode, the performance can be surpassed after the first batch of incremental data arrives. In comparison, the performance improvement is more significant when the difference between the initial number of nodes and the number of incremental nodes is large. For example, when the initial number of nodes is set to 100, the final performance of the data-driven dynamic node mechanism is improved by 2% compared to the standard BLS and static incremental BLS.

[0074] It should also be noted that, in this specification, terms such as "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data flow-driven dynamic node width learning image classification method, characterized in that, Includes the following steps: S1. Set the number of initial mapping feature nodes and enhancement nodes, input the initial batch image data, and build a width learning model based on the set parameters; S2. Determine whether a new image data stream has arrived or whether the preset termination condition has not been met. If not, output the trained width learning model and test its performance; if yes, proceed to step S3. S3. Calculate the number of enhancement nodes that need to be added; S4. Adjust the node configuration of the width learning model, update the weights of the width learning model using incremental data stream, and jump to step S2. In step S3, there are two methods to calculate the number of enhancement nodes that need to be added, specifically: The number of augmentation nodes in the model is adjusted based on the current training sample data used for width learning and the number of samples in the new data stream. The new number of nodes, m, is... ′ (t+1) is calculated using formula (6): Where m(t) is the number of augmentation nodes at the current time, im(t) is the number of samples in the new data stream, and l(t) is the number of samples in the training data used by the model at the current time. It's a floor operation; the number of enhancement nodes needed, Δm, is... Based on the current number of augmentation nodes m(t) in the width learning model, adjust the number of augmentation nodes in the model, resulting in a new number m. ′ (t+1) is calculated using formula (7): The number of enhancement nodes that need to be added, Δm, is...

2. The dynamic node width learning image classification method according to claim 1, characterized in that, Step S1, which involves establishing a width learning model based on the set parameters, specifically includes the following steps: Using the initial image input X, calculate the feature map node Z. n ≡[Z1,Z2,…,Z n The process is a linear mapping, where the i-th group of mapping features Z i The calculation is as shown in formula (1), and the connection weight W ei and β ei It is generated randomly. For linear mappings, each set of features contains k mapping features; By mapping node Z n Calculate the enhancement node H m ≡[H1,H2,…,H m This process is a non-linear mapping, where the enhanced feature H... j The calculation is as shown in formula (2), and the connection weight W hj and β hj It is randomly generated, ζ(Z,W) hj ,β hj () is a non-linear activation function; H j ≡ζ(Z n W hj +β hj ),j=1,2,…,m (2) The mapping node represents the linear characteristics of the data, and the enhancement node represents the nonlinear characteristics of the data; Linear and nonlinear features are combined into a feature layer A = [Z|H], which is then connected to the model output layer Y. Since the information of the model's output layer Y is known, i.e., the category corresponding to the data, we only need to calculate the connection weights W between the feature layer and the output layer. The calculation formula is as follows: W=A -1 Y (3) In the calculation, ridge regression is used to solve for the weight matrix, that is, W is solved by the optimization problem of formula (3): Where σ1=σ2=v=u=2, we can solve for: W=(λI+AA T ) -1 A T Y(5)。 3. The dynamic node width learning image classification method according to claim 1, characterized in that, In step S2, the preset termination condition is that the test accuracy of the width learning model reaches the preset test accuracy.

4. The dynamic node width learning image classification method according to claim 1, characterized in that, In step S4, updating the weights of the width learning model using incremental data streams specifically includes retraining and incremental learning methods.

5. The dynamic node width learning image classification method according to claim 4, characterized in that, The retraining method is as follows: The previously used training image data X(t) is merged with the newly arrived incremental data stream B(t+1) to form new training data X(t+1) = [X(t)|(t+1)]; Based on the new training data X(t+1), calculate the feature mapping node Z. n (t+1); Based on the calculated mapped feature nodes and the adjusted number of enhanced nodes m ′ (t+1) Calculate the enhanced feature H m (t+1), merge the mapped features and enhanced features into a feature layer A(t+1) = [Z|H], and connect it to the output layer Y of the width learning model; The output layer connection weights are calculated using formulas (3) and (4).

6. The dynamic node width learning image classification method according to claim 4, characterized in that, The incremental learning method is as follows: Based on the newly arrived incremental data stream B(t+1), calculate the corresponding feature A according to the current model parameter settings. x As shown in formula (8): in, It is the mapping node corresponding to the incremental data; W ei β ei W hj and β ej The weights of the model in the t-th batch of data; The corresponding new features at this time Calculated using formula (9): At the same time, it is also necessary to consider the current mapping characteristics. Calculate the enhancement feature H corresponding to adding Δm enhancement nodes. Δm ≡[H1,H2,…,H Δm ], where the j-th enhancement feature H j Calculate using formula (10): The features after adding data and enhancing nodes are calculated as shown in formula (11): The model weights W(t) obtained based on the previous training data X(t), and the connection weights W between the features generated by the incremental samples and the output layer. x (t+1) is calculated directly from formula (5); The new weights W(t+1) = [W(t)|W x [(t+1)]; After adding the enhancement node, the final weights of the model are updated using formula (12): Among them, Y x These are the labels for the incremental data B(t+1).

Images