A dual-branch model training method for CNN inference in the Internet of Things

Abstract

This invention belongs to the field of IoT computing technology, specifically relating to a dual-branch model training method for IoT CNN inference. The method includes: constructing a dual-branch CNN training model, where the lower branch is an image classification model and the upper branch is obtained by adding a loss layer after the convolutional layer of the image classification model; training the dual-branch CNN training model and saving the image classification model of the lower branch; deploying the image classification model to IoT edge nodes and selecting the optimal transmission loop based on the model's weight parameters and the node's location information; assigning the inference task to the initial node, which then performs inference according to the optimal transmission loop to obtain the image classification result. This invention ensures that the complexity of the inference model remains unchanged while improving inference performance and increasing the efficiency of IoT communication inference.

Description

Technical Field

[0001] This invention belongs to the field of Internet of Things (IoT) computing technology, specifically relating to a dual-branch model training method for IoT CNN inference. Background Technology

[0002] Convolutional Neural Networks (CNNs) are one of the core technologies in the field of deep learning. Especially in areas such as image recognition, object detection, and speech recognition, CNNs have become standard tools. However, with the expansion of application scenarios and changes in requirements, CNNs face some challenges, such as computational resource consumption, model efficiency, and generalization ability, prompting numerous researchers to continuously explore and optimize them.

[0003] With the rapid development of the Internet of Things (IoT) and edge computing technologies, massive amounts of data, especially image and video data, have been generated. While traditional cloud computing offers powerful computing capabilities, it suffers from limitations in real-time performance, bandwidth, privacy protection, and energy efficiency. Particularly in applications such as smart homes, smart surveillance, and autonomous driving, processing this data closer to the data source has become a pressing issue. The introduction of edge computing, combined with convolutional neural networks (CNNs), can migrate data processing from the cloud to the network edge, significantly improving system real-time performance, reducing bandwidth requirements, and enhancing data privacy protection. Despite the powerful feature learning capabilities of CNNs, their high computational complexity and resource consumption during the inference phase remain a major challenge in deployment and application. With the diversification of application scenarios, improving CNN inference efficiency and reducing its computational and memory overhead has become a current research hotspot in the field of deep learning.

[0004] To address the excessive resource demands of convolutional neural networks (CNNs), traditional methods rely on powerful cloud data centers for intensive deep neural network computations. In this scenario, input data generated from mobile devices is sent to a remote cloud data center, and the device receives the execution results upon completion of the computation. However, using this cloud-centric approach, large amounts of data (such as images and videos) are transmitted back and forth between terminal devices and remote cloud data centers over long-range wide area networks (WANs), potentially leading to intolerable latency and wasted energy. To address this issue, existing technologies have designed a low-latency collaborative inference framework for device-edge collaboration. This framework relies on DNN partitioning, adaptively partitioning DNN computations between mobile devices and edge servers based on available bandwidth to leverage the computing power of the edge servers.

[0005] However, these technologies often neglect improvements to the CNN model itself and the design of communication schemes, resulting in low communication and inference efficiency for IoT devices. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a dual-branch model training method for CNN inference in the Internet of Things (IoT). This method includes:

[0007] S1: Construct a dual-branch CNN training model, with the lower branch being an image classification model and the upper branch being obtained by adding a loss layer after the convolutional layer of the image classification model;

[0008] S2: Train the dual-branch CNN training model and save the image classification model of the lower branch;

[0009] S3: Deploy the image classification model to IoT edge nodes and select the optimal transmission loop based on the model's weight parameters and the node's location information;

[0010] S4: Assign the inference task to the initial node. The initial node performs inference according to the optimal transmission loop to obtain the output image classification result.

[0011] Preferably, during the training of the dual-branch CNN training model, if the feature map size changes after the upper branch convolution, the output of the loss layer before that convolutional layer is superimposed with the output of the convolutional layer at the corresponding position of the lower branch.

[0012] Preferably, the loss layer is represented as follows:

[0013] f loss =V i ·L+n

[0014] Among them, f loss V represents the output of the loss layer. i Let L represent the output of the i-th convolutional layer, L represent the Rayleigh distribution, and n represent additive white Gaussian noise.

[0015] Preferably, the image classification model includes 13 convolutional layers, each of which is activated by the ReLU activation function and connected to a BN layer; the 2nd, 4th, 7th, 10th, and 13th convolutional layers are followed by max pooling layers; the last convolutional layer is followed by three fully connected layers, the first two of which are activated by the ReLU activation function, and the last fully connected layer is connected to the Softmax function.

[0016] Preferably, the process of selecting the optimal transmission loop based on the model's weight parameters and node location information includes:

[0017] Obtain the total set of transmission loops and calculate the evaluation factor for each node;

[0018] Count the number of IoT edge nodes. If the number of nodes is sufficient, find the transmission loop that minimizes the sum of evaluation factors from the total set of transmission loops.

[0019] If the number of nodes is insufficient, then find the subset of transmission loops that minimizes the sum of evaluation factors from the total set of transmission loops.

[0020] Preferably, the formula for the evaluation factor of the computing node is:

[0021]

[0022] Where, ε i Let d represent the evaluation factor of the i-th node. i,j Let represent the Euclidean distance from the i-th node to the j-th node. This represents the average weight of the image classification model at the i-th node.

[0023] Furthermore, if the number of nodes is sufficient, the number of nodes in the transmission loop that minimizes the sum of evaluation factors is equal to the number of convolutional layers in the image classification model.

[0024] Furthermore, if the number of nodes is insufficient, the subset of transmission loops that minimizes the sum of evaluation factors contains 'a' loops, and the sum of the node counts in these 'a' loops equals the number of convolutional layers in the image classification model; where:

[0025]

[0026] Where m represents the number of convolutional layers in the image classification model, and K represents the number of IoT edge nodes. This indicates rounding up to the nearest integer.

[0027] The beneficial effects of this invention are as follows: This invention designs a dual-branch training model that only increases network complexity during the training phase. The aim is to enhance the model's noise and fading resistance during the pre-training phase through the dual-branch model. The lower branch uses the original network model, while the upper branch simulates multiplicative fading of the channel through a loss layer. After the feature map is simultaneously input into both branches, the upper branch superimposes the training data of the convolutional layers with modified feature map sizes onto the lower branch to simulate additive noise in the channel. The lower branch trains the model according to the original network structure. This model only increases model complexity during the pre-training phase. When IoT devices perform inference, the original offline-unloaded network model is still used, ensuring that the inference model complexity remains unchanged while improving inference performance. This invention combines a node selection algorithm, integrating model weights with transmission distance to select the optimal transmission loop, thereby improving the efficiency of IoT communication inference. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the dual-branch CNN training model structure in this invention;

[0029] Figure 2 This is a schematic diagram illustrating the implementation of the node selection algorithm in this invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] This invention proposes a dual-branch model training method for CNN inference in the Internet of Things, the method comprising:

[0032] S1: Construct a dual-branch CNN training model, with the lower branch being an image classification model and the upper branch being obtained by adding a loss layer after the convolutional layer of the image classification model.

[0033] like Figure 1 As shown, the dual-branch CNN training model includes two branches: the lower branch is an image classification model, and the upper branch is obtained by adding a loss layer after the convolutional layer of the image classification model. The dual-branch CNN training model is only used for training. During training, if the feature map size changes after the convolution of the upper branch, the output of the loss layer before the convolutional layer is superimposed with the output of the convolutional layer at the corresponding position of the lower branch.

[0034] S2: Train the dual-branch CNN training model and save the image classification model of the lower branch.

[0035] The trained image classification model retains the original network structure. Preferably, the model includes 13 convolutional layers, each with a 3×3 kernel and a stride of 1, applying the same padding to ensure identical input and output sizes. Each convolutional layer is followed by a ReLU activation function and connected to a Batch Normalization (BN) layer. The 2nd, 4th, 7th, 10th, and 13th convolutional layers are followed by max pooling layers with a 2×2 window and a stride of 2. The last convolutional layer is followed by three fully connected layers: two 4096-neuron fully connected layers and one 1000-neuron fully connected layer. The first two fully connected layers use ReLU activation, and the last fully connected layer uses a Softmax function. The BN layer accelerates network convergence and improves gradient vanishing. The ReLU activation function after the convolutional layers performs non-linear data transformation, while the Softmax activation function after the fully connected layers is more suitable for multi-class classification tasks requiring inference.

[0036] In the upper branch, considering that the CNN inference process runs through different convolutional layers at different nodes, a loss layer is added after each convolutional layer to simulate the small-scale fading and noise effects encountered during data transmission in node communication. The loss layer can be represented as:

[0037] f loss =V i ·L+n

[0038] Among them, f loss V represents the output of the loss layer. i Let L represent the output of the i-th convolutional layer, L represent the Rayleigh distribution, and n represent the distribution with mean 0 and variance ω. 2 Additive white Gaussian noise.

[0039] When the size of the feature maps in the network changes, the upper branch simulates multiplicative Rayleigh fading through a loss layer, and the output of the upper branch loss layer is superimposed with the output of the corresponding convolutional layer in the lower branch to simulate additive noise effects. This process not only enhances the model's robustness to input noise but also allows the model to maintain high performance when facing channel fading. Ultimately, the trained model can effectively resist interference and achieve more accurate signal recovery and processing. It is worth noting that because this invention uses a design that superimposes the weight data trained in the upper branch onto the corresponding lower branch, the model complexity only increases during the training phase. The advantage of this design strategy is that in the actual communication inference phase, the traditional model of the lower branch can still be used for inference. This choice ensures that the model complexity of the communication inference process remains unchanged in practical applications, guaranteeing the system's efficiency.

[0040] S3: Deploy the image classification model to IoT edge nodes and select the optimal transmission loop based on the model's weight parameters and the node's location information.

[0041] The node selection algorithm of this invention requires that the same task node appear only once in a transmission loop, and does not require coverage of every node in the Internet of Things system.

[0042] Since node data transmission is affected by path fading in each communication, a greedy algorithm can be considered to find the optimal solution for the current state of a node, thereby obtaining the nearest next node. Assuming a single inference task with m convolutional layers in the network model, at least m data transmissions are required, and d... i,j Let ε represent the Euclidean distance from node i to node j, and therefore define the evaluation factor ε for node i. i :

[0043]

[0044] in, This represents the average weight of the image classification model at the i-th node.

[0045] like Figure 2 As shown, obtain the total set of transmission loops, and select the most suitable transmission loop from the total set of transmission loops. Specifically:

[0046] Count the number of IoT edge nodes, and consider two scenarios:

[0047] 1) If the number of nodes K is sufficient (i.e., K is not less than m), the entire inference process can be completed within a single transmission loop. The objective function can then be expressed as:

[0048]

[0049] The above formula means that if the number of nodes is sufficient, then the transmission loop that minimizes the sum of evaluation factors is found from the total set of transmission loops. The number of nodes in the transmission loop that minimizes the sum of evaluation factors is equal to the number of convolutional layers in the image classification model.

[0050] 2) If the number of nodes K is insufficient (K < m), multiple transmission loops need to be selected to complete the entire inference process; the objective function can be rewritten as:

[0051]

[0052] Where, ε c Let C represent the sum of evaluation factors for all nodes in loop c, where C = {c2, c3, ..., c...} K},c K This represents the shortest loop in the system consisting of K nodes, and stores the transmission distance of all nodes within that loop. pot This represents the optimal set of transmission loops.

[0053] The above formula means that if the number of nodes is insufficient, the subset of transmission loops that minimizes the sum of evaluation factors is found from the total set of transmission loops. pot The number of Middle Ring Roads is calculated as follows:

[0054]

[0055] Where a represents T pot The number of convolutional layers in the image classification model represents the minimum number of convolutional layers required to complete one inference task. Here, m represents the number of convolutional layers in the model, and K represents the number of edge nodes in the Internet of Things (IoT) system. This indicates rounding up to the nearest integer.

[0056] Set T pot The total number of nodes in the model equals the number of convolutional layers in the image classification model; for example, if a = 3, m = 10, T potThe elements in can be combinations of c2, c3, c5 or c3, c3, c4, etc.

[0057] S4: Assign the inference task to the initial node. The initial node performs inference according to the optimal transmission loop to obtain the image classification result.

[0058] In summary, this invention designs a dual-branch training model that increases network complexity only during the training phase. During training, the traditional network model is extended to a dual-branch structure; the lower branch retains the original model structure; the newly extended branch adds a loss layer to the lower branch for training, and the training results of each layer of the upper branch are superimposed on the lower branch, resulting in a model that can resist noise and fading during node communication. The inference model only needs to use the original branch's network structure, without increasing the complexity of the inference model. Furthermore, a node selection algorithm based on model weights is incorporated. Since weights affect signal transmission power, nodes with longer transmission distances require higher transmission power to resist path loss; therefore, the average weight and path are considered as evaluation parameters for node selection to obtain the optimal transmission path and ensure transmission efficiency.

[0059] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for training a dual-branch model for CNN inference in the Internet of Things, characterized in that, include: S1: Construct a dual-branch CNN training model, with the lower branch being an image classification model and the upper branch being obtained by adding a loss layer after the convolutional layer of the image classification model; S2: Train the dual-branch CNN training model and save the image classification model of the lower branch; during the training of the dual-branch CNN training model, if the feature map size changes after the upper branch convolution, the output of the loss layer before the convolutional layer is superimposed with the output of the convolutional layer at the corresponding position of the lower branch. S3: Deploy the image classification model to IoT edge nodes and select the optimal transmission loop based on the model's weight parameters and the node's location information; the process of selecting the optimal transmission loop based on the model's weight parameters and the node's location information includes: Obtain the total set of transmission loops and calculate the evaluation factor for each node; the formula for calculating the evaluation factor of a node is: Where, ε i Let d represent the evaluation factor of the i-th node. i,j Let represent the Euclidean distance from the i-th node to the j-th node. This represents the average weight of the image classification model at the i-th node; Count the number of IoT edge nodes. If the number of nodes is sufficient, find the transmission loop that minimizes the sum of evaluation factors from the total set of transmission loops. If the number of nodes is insufficient, then find the subset of transmission loops that minimizes the sum of evaluation factors from the total set of transmission loops; S4: Assign the inference task to the initial node. The initial node performs inference according to the optimal transmission loop to obtain the image classification result.

2. The method for training a dual-branch model for CNN inference in the Internet of Things according to claim 1, characterized in that, The loss layer is represented as: f loss =V i ·L+n Among them, f loss V represents the output of the loss layer. i Let L represent the output of the i-th convolutional layer, L represent the Rayleigh distribution, and n represent additive white Gaussian noise.

3. The method for training a dual-branch model for CNN inference in the Internet of Things according to claim 1, characterized in that, The image classification model consists of 13 convolutional layers. Each convolutional layer is activated by the ReLU activation function and connected to a BN layer. The 2nd, 4th, 7th, 10th, and 13th convolutional layers are followed by max pooling layers. The last convolutional layer is followed by three fully connected layers. The first two fully connected layers are activated by the ReLU activation function, and the last fully connected layer is connected to the Softmax function.

4. The method for training a dual-branch model for CNN inference in the Internet of Things according to claim 1, characterized in that, If there are enough nodes, the number of nodes in the transmission loop that minimizes the sum of evaluation factors is equal to the number of convolutional layers in the image classification model.

5. The method for training a dual-branch model for CNN inference in the Internet of Things according to claim 1, characterized in that, If the number of nodes is insufficient, the subset of transmission loops that minimizes the sum of evaluation factors contains 'a' loops, and the sum of the number of nodes in these 'a' loops equals the number of convolutional layers in the image classification model; where: Where m represents the number of convolutional layers in the image classification model, and K represents the number of IoT edge nodes. This indicates rounding up to the nearest integer.

Images