Double-branch model training method for CNN reasoning of Internet of Things

Abstract

The invention belongs to the technical field of computing of the Internet of Things, and particularly relates to a double-branch model training method for CNN reasoning of the Internet of Things. The method comprises the steps that a double-branch CNN training model is constructed, a lower branch is an image classification model, and an upper branch is obtained by adding a loss layer behind a convolutional layer of the image classification model; training a double-branch CNN training model, and storing an image classification model of a lower branch; deploying the image classification model to an edge node of the Internet of Things, and selecting an optimal transmission loop according to a weight parameter of the model and position information of the node; the reasoning task is distributed to the initial node, the initial node conducts reasoning according to the optimal transmission loop, and an image classification result is obtained; according to the method, the reasoning performance is improved while the complexity of the reasoning model is not changed, and the efficiency of communication reasoning of the Internet of Things is improved.

Description

Technical Field

[0001] The present invention belongs to the field of Internet of Things computing technology, and in particular relates to a dual-branch model training method for CNN reasoning in the Internet of Things. Background Art

[0002] Convolutional neural network (CNN) is one of the core technologies in the field of deep learning. Especially in the fields of image recognition, object detection, speech recognition, etc., CNN has become a standard tool. However, with the expansion of application scenarios and changes in demand, some problems faced by CNN, such as computing resource consumption, model efficiency, generalization ability, etc., have prompted many researchers to continuously explore and optimize.

[0003] With the rapid development of the Internet of Things (IoT) and edge computing technologies, a large amount of data, especially image and video data, has been generated. Although the traditional cloud computing model can provide powerful computing power, it has many limitations in terms of real-time performance, bandwidth, privacy protection, and energy efficiency. Especially in application scenarios such as smart homes, smart monitoring, and autonomous driving, how to process this data close to the data source has become an urgent issue. The introduction of edge computing, combined with convolutional neural networks (CNNs), can migrate data processing from the cloud to the edge of the network, significantly improving the real-time performance of the system, reducing bandwidth requirements, and improving data privacy protection capabilities. Although CNN has powerful feature learning capabilities, its high computational complexity and resource consumption in the reasoning stage are still a major challenge in deployment and application. With the diversification of application scenarios, how to improve the efficiency of CNN reasoning and reduce its computational and memory overhead has become a research hotspot in the current deep learning field.

[0004] To cope with the excessive demand for resources of convolutional neural networks, traditional approaches use powerful cloud data centers to perform intensive computations of deep neural networks. In this case, input data generated from mobile devices is sent to remote cloud data centers, and the device receives the execution results when the computation is completed. However, using this cloud-centric approach, large amounts of data (such as images and videos) will be transmitted back and forth between terminal devices and remote cloud data centers over long wide area networks, which may cause unbearable delays and waste energy. To address this problem, the prior art designs a low-latency collaborative reasoning framework for device-edge collaboration, relying on DNN partitioning to adaptively partition DNN computations between mobile devices and edge servers based on available bandwidth to utilize the computing power of edge servers.

[0005] However, these technologies often ignore the improvement of the CNN model itself and the design of communication schemes, resulting in low communication and reasoning efficiency of IoT devices. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a dual-branch model training method for CNN reasoning in the Internet of Things, the method comprising:

[0007] S1: Construct a dual-branch CNN training model, where the lower branch is an image classification model and the upper branch is obtained by adding a loss layer after the convolution layer of the image classification model;

[0008] S2: Train the dual-branch CNN training model and save the image classification model of the lower branch;

[0009] S3: Deploy the image classification model to the IoT edge node and select the optimal transmission loop based on the model weight parameters and node location information;

[0010] S4: Assign the reasoning task to the initial node, and the initial node performs reasoning according to the optimal transmission loop to obtain the output image classification result.

[0011] Preferably, during the training process of the dual-branch CNN training model, after the convolution of the upper branch, if the size of the feature map changes, the output of the loss layer before the convolution layer is superimposed with the output of the convolution layer at the corresponding position of the lower branch.

[0012] Preferably, the loss layer is expressed as:

[0013] f loss =V i ·L+n

[0014] Among them, f loss represents the loss layer output, V i represents the output of the i-th convolutional layer, L represents Rayleigh distribution, and n represents additive Gaussian white noise.

[0015] Preferably, the image classification model includes 13 convolutional layers, after each convolutional layer, a BN layer is activated and connected using the ReLU activation function; the 2nd, 4th, 7th, 10th, and 13th convolutional layers are connected to the maximum pooling layer; the last convolutional layer is connected to three fully connected layers, the first two fully connected layers are activated using the ReLU activation function, and the last fully connected layer is connected to the Softmax function.

[0016] Preferably, the process of selecting the optimal transmission loop according to the weight parameters of the model and the location information of the nodes includes:

[0017] Obtain the total set of transmission loops and calculate the evaluation factor of each node;

[0018] Count the number of IoT edge nodes. If the number of nodes is sufficient, find a transmission loop with the smallest sum of evaluation factors from the total set of transmission loops.

[0019] If the number of nodes is insufficient, a subset of transmission loops with the smallest sum of evaluation factors is sought from the total set of transmission loops.

[0020] Preferably, the formula for calculating the evaluation factor of the node is:

[0021]

[0022] Among them, ε i represents the evaluation factor of the i-th node, d i,j represents the Euclidean distance from the i-th node to the j-th node, Represents the mean weight of the image classification model on the i-th node.

[0023] Furthermore, if the number of nodes is sufficient, the number of nodes in a transmission loop with the smallest sum of evaluation factors is equal to the number of convolutional layers in the image classification model.

[0024] Furthermore, if the number of nodes is insufficient, the number of loops in the transmission loop subset with the smallest sum of evaluation factors is a, and the sum of the number of nodes in a loops is equal to the number of convolutional layers in the image classification model; where:

[0025]

[0026] Where m represents the number of convolutional layers in the image classification model, K represents the number of IoT edge nodes, Indicates rounding up.

[0027] The beneficial effects of the present invention are as follows: the present invention designs a dual-branch training model that only increases the network complexity in the training stage, and aims to increase the anti-noise and anti-fading capabilities of the model through the dual-branch model in the pre-training stage. The lower branch adopts the original network model, and the upper branch models the multiplicative fading of the channel through the Loss layer. After the feature map is simultaneously input into the upper and lower branches, the upper branch will superimpose the training data of the convolutional layer with the changed feature map size onto the lower branch to simulate the additive noise of the channel, while the lower branch trains the model according to the original network structure. This model only increases the model complexity in the pre-training stage, and when the IoT device performs reasoning, the original network model that is unloaded offline is still used, which ensures that the complexity of the reasoning model remains unchanged while improving the reasoning performance. The present invention combines the node selection algorithm, combines the model weight with the transmission distance, selects the optimal transmission loop, and improves the efficiency of IoT communication reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the dual-branch CNN training model structure in the present invention;

[0029] Figure 2 This is a schematic diagram of the node selection algorithm implementation in the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] The present invention proposes a dual-branch model training method for CNN reasoning in the Internet of Things, the method comprising:

[0032] S1: Construct a dual-branch CNN training model. The lower branch is an image classification model, and the upper branch is obtained by adding a loss layer after the convolution layer of the image classification model.

[0033] like Figure 1 As shown in the figure, the dual-branch CNN training model includes two branches, the lower branch is the image classification model, and the upper branch is obtained by adding a loss layer after the convolution layer of the image classification model; the dual-branch CNN training model is only used for training. During training, after the convolution of the upper branch, if the size of the feature map changes, the output of the loss layer before the convolution layer is superimposed with the output of the convolution layer at the corresponding position of the lower branch.

[0034] S2: Train the dual-branch CNN training model and save the image classification model of the lower branch.

[0035] After training, the image classification model still maintains the original network structure. Preferably, the image classification model includes 13 convolutional layers, each with a convolution kernel of 3×3 and a step size of 1, and the same padding is applied to ensure the same input and output size; after each convolutional layer, a BN (batch normalization) layer is activated and connected using the ReLU activation function; the second, fourth, seventh, tenth, and thirteenth convolutional layers are connected to the maximum pooling layer, and the maximum pooling window is 2×2 and the step size is 2; the last convolutional layer is connected to three fully connected layers, namely, two fully connected layers of 4096 neurons and one fully connected layer of 1000 neurons; the first two fully connected layers are activated using the ReLU activation function, and the last fully connected layer is connected to the Softmax function. The role of the BN layer is to accelerate network convergence and improve gradient diffusion. The activation function layer after the convolutional layer uses the Relu function to perform nonlinear transformation of data. The activation function after the full connection uses Softmax, which is more suitable for multi-classification tasks of reasoning.

[0036] In the upper branch, considering that the CNN reasoning process will run different convolutional layers at different nodes, a Loss layer is added after each convolutional layer to simulate the small-scale fading and noise effects encountered during data transmission during node communication. The Loss layer can be expressed as:

[0037] f loss =V i ·L+n

[0038] Among them, f loss represents the loss layer output, V i represents the output of the i-th convolutional layer, L represents Rayleigh distribution, and n represents a mean of 0 and a variance of ω 2 Additive white Gaussian noise.

[0039] When the size of the feature map in the network changes, the upper branch simulates multiplicative Rayleigh fading through the loss layer, and superimposes the output of the upper branch loss layer with the convolution layer output corresponding to the lower branch to simulate the additive noise effect. This process not only enhances the robustness of the model to input noise, but also enables the model to maintain high performance in the face of channel fading. Ultimately, the trained model can effectively resist interference and achieve more accurate signal recovery and processing. It is worth noting that since the present invention adopts a design that superimposes the weight data of the upper branch training to the corresponding lower branch, the model complexity is only increased in the training stage. The advantage of this design strategy is that in the actual communication reasoning stage, the traditional model of the lower branch can still be used for reasoning. This choice ensures that in practical applications, the model complexity of the communication reasoning process remains unchanged, ensuring the efficiency of the system.

[0040] S3: Deploy the image classification model to the IoT edge node and select the optimal transmission loop based on the model’s weight parameters and the node’s location information.

[0041] The node selection algorithm of the present invention requires that the same task node only appears once in a transmission loop, and does not require covering every node in the Internet of Things system.

[0042] Since the transmission of node data is affected by path fading in each communication, a greedy algorithm can be considered to solve the optimal solution for the current state of the node, so as to obtain the nearest next node. Assuming that for an inference task, the network model has m convolutional layers, then at least m data transmissions need to be completed, with d i,j Represents the Euclidean distance from node i to node j, so the evaluation factor ε of node i is defined i :

[0043]

[0044] in, Represents the mean weight of the image classification model on the i-th node.

[0045] like Figure 2 As shown, a total set of transmission loops is obtained, and the most suitable transmission loop is selected from the total set of transmission loops. Specifically:

[0046] Count the number of IoT edge nodes and consider two situations:

[0047] 1) The number of nodes K is sufficient, that is, K is not less than m, and the entire reasoning process can be completed in one transmission loop. The objective function can be expressed as:

[0048]

[0049] The above formula means: if the number of nodes is sufficient, then a transmission loop with the smallest sum of evaluation factors is found from the total set of transmission loops. The number of nodes in a transmission loop with the smallest sum of evaluation factors is equal to the number of convolutional layers in the image classification model.

[0050] 2) If the number of nodes K is insufficient, that is, K is less than m, multiple transmission loops need to be selected to complete the entire reasoning process; the objective function can be rewritten as:

[0051]

[0052] Among them, ε c represents the sum of the evaluation factors of all nodes in the loop c, C = {c2, c3, ..., c K},c K Indicates that the system consists of K nodes forming the shortest loop and saves the transmission distance of all nodes in the loop. pot Represents the optimal subset of transmission rings.

[0053] The above formula indicates that if the number of nodes is insufficient, then a subset of transmission loops with the smallest sum of evaluation factors is found from the total set of transmission loops. pot The number of loops is calculated as follows:

[0054]

[0055] Where a represents T pot The number of loops in the model is the minimum number of loops required to complete an inference task, m represents the number of convolutional layers in the image classification model, and K represents the number of IoT edge nodes. Indicates rounding up.

[0056] Collection T pot The sum of the number of nodes in the network is equal to the number of convolutional layers in the image classification model; for example, if a = 3, m = 10, T potThe elements in can be a combination of c2, c3, c5 or c3, c3, c4 and so on.

[0057] S4: Assign the reasoning task to the initial node, and the initial node performs reasoning according to the optimal transmission loop to obtain the image classification result.

[0058] In summary, the present invention designs a dual-branch training model that only increases the complexity of the network during the training phase. During the training phase, the traditional network model is expanded into a dual-branch structure; the lower branch maintains the original structure of the model; the newly expanded branch adds a loss layer for training on the basis of the lower branch, and the training results of each layer of the upper branch are superimposed on the lower branch to obtain a model that can resist noise and fading during node communication, while the inference model only needs to use the network structure of the original branch, and does not increase the complexity of the inference model. Combined with the node selection algorithm based on model weights, since the weight affects the transmission power of the signal, nodes with a longer transmission distance require higher transmission power to resist path loss; therefore, the weight mean and path are considered as evaluation parameters for node selection to obtain the optimal transmission path and ensure transmission efficiency.

[0059] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A dual-branch model training method for CNN reasoning in the Internet of Things, characterized in that: include: S1: Construct a dual-branch CNN training model, where the lower branch is an image classification model and the upper branch is obtained by adding a loss layer after the convolution layer of the image classification model; S2: Train the dual-branch CNN training model and save the image classification model of the lower branch; S3: Deploy the image classification model to the IoT edge node and select the optimal transmission loop based on the model’s weight parameters and the node’s location information; S4: Assign the reasoning task to the initial node, and the initial node performs reasoning according to the optimal transmission loop to obtain the image classification result.

2. According to claim 1, a dual-branch model training method for CNN reasoning in the Internet of Things is characterized in that: During the training process of the dual-branch CNN training model, after the convolution of the upper branch, if the size of the feature map changes, the output of the loss layer before the convolution layer is superimposed with the output of the convolution layer at the corresponding position of the lower branch.

3. A dual-branch model training method for CNN reasoning in the Internet of Things according to claim 1, characterized in that: The loss layer is expressed as: f loss =V i ·L+n Among them, f loss represents the loss layer output, V i represents the output of the i-th convolutional layer, L represents Rayleigh distribution, and n represents additive Gaussian white noise.

4. A dual-branch model training method for CNN reasoning in the Internet of Things according to claim 1, characterized in that: The image classification model consists of 13 convolutional layers. After each convolutional layer, a BN layer is activated and connected using the ReLU activation function. The 2nd, 4th, 7th, 10th, and 13th convolutional layers are connected to the maximum pooling layer. The last convolutional layer is connected to three fully connected layers. The first two fully connected layers are activated using the ReLU activation function, and the last fully connected layer is connected to the Softmax function.

5. A dual-branch model training method for CNN reasoning in the Internet of Things according to claim 1, characterized in that: The process of selecting the optimal transmission loop according to the weight parameters of the model and the location information of the nodes includes: Obtain the total set of transmission loops and calculate the evaluation factor of each node; Count the number of IoT edge nodes. If the number of nodes is sufficient, find a transmission loop with the smallest sum of evaluation factors from the total set of transmission loops. If the number of nodes is insufficient, a subset of transmission loops with the smallest sum of evaluation factors is sought from the total set of transmission loops.

6. A dual-branch model training method for CNN reasoning in the Internet of Things according to claim 5, characterized in that: The formula for calculating the evaluation factor of a node is: Among them, ε i represents the evaluation factor of the i-th node, d i,j represents the Euclidean distance from the i-th node to the j-th node, Represents the mean weight of the image classification model on the i-th node.

7. A dual-branch model training method for CNN reasoning in the Internet of Things according to claim 5, characterized in that: If the number of nodes is sufficient, the number of nodes in a transmission loop with the smallest sum of evaluation factors is equal to the number of convolutional layers in the image classification model.

8. A dual-branch model training method for CNN reasoning in the Internet of Things according to claim 5, characterized in that: If the number of nodes is insufficient, the number of loops in the transmission loop subset with the smallest sum of evaluation factors is a, and the sum of the number of nodes in a loops is equal to the number of convolutional layers in the image classification model; where: Where m represents the number of convolutional layers in the image classification model, K represents the number of IoT edge nodes, Indicates rounding up.

Images