A neural network model training method based on edge computing

Abstract

The present invention provides a neural network model training method based on edge computing. The method includes: the edge side uses a deep learning model based on a non-learning layer to train the collected video surveillance data on a video analysis model to obtain model parameters, and sends the model parameters to the terminal device; the terminal device constructs a video inference model by using the deep learning model of the non-learning layer based on the model parameters distributed by the edge side, and uses the video inference model to perform real-time analysis on the collected video surveillance data, and uploads the video surveillance data and the analysis results to the edge side. The present invention proposes to improve the model in a non-learning layer manner in video analysis tasks based on edge computing and deep learning, optimize data transmission between the edge side and the terminal device, reduce communication latency, and at the same time maintain high-precision real-time video analysis capabilities.

Description

Technical Field

[0001] The present invention relates to the technical field of neural network training, and in particular to a method for training a neural network model based on edge computing. Background Art

[0002] With the advent of the era of Internet of Everything and 5G, more and more terminal devices are deployed in key scenarios and show an intelligent development trend. Providing convenient and efficient services for users through neural network models has become the current mainstream mode. However, in actual scenarios, the limited resources on terminal devices will reduce the computing efficiency of neural network models, resulting in a large delay, thus making this computing mode lose its applicability. In recent years' edge computing, decomposing and allocating computing tasks, and the collaborative training mode through multi-terminal, edge-side, and cloud devices has effectively reduced the delay.

[0003] Currently, the existing cloud / edge-end collaborative training methods for deep neural networks include: considering that their usage scenarios require analyzing visual information, convolutional networks are generally used to construct the model, and the model is optimized on this basis. Separable convolution is a relatively effective solution. By decomposing the convolutional kernel into matrices, the two-dimensional convolutional kernel is split into two one-dimensional convolutional kernels, significantly reducing the number of parameters in model training and the amount of data transmitted between cloud / edge-end devices, and alleviating the delay caused by model training and parameter transmission to a certain extent.

[0004] The disadvantages of the above existing cloud / edge-end collaborative training methods for deep neural networks are as follows: The optimization of these methods for neural network models reduces the number of parameters to a certain extent, but these methods still have two limitations. First, the reduction in the number of parameters of the model does not significantly improve real-time performance. Theoretically, separable convolution reduces the convolutional kernel parameters by an order of magnitude, but when the convolutional kernel itself is small, such as decomposing a 3×3 convolutional kernel into two 3×1 convolutional kernels, the overall number of parameters does not significantly decrease. In addition, the number of parameters is still affected by the depth of the model, and the training tasks and network transmission load have not been well optimized.

[0005] Second, the structural optimization does not provide an advantage in training. Since current GPU (graphics processing unit) and TPU (tensor processing unit) devices focus on accelerating and optimizing traditional convolutional kernels in parallel training, the reduction in the number of parameters brought by separable convolution does not truly reflect the efficiency improvement in the training and inference processes. Summary of the Invention

[0006] An embodiment of the present invention provides a method for training a neural network model based on edge computing to optimize data transmission between the edge side and terminal devices.

[0007] To achieve the above object, the present invention adopts the following technical solutions.

[0008] A method for training a neural network model based on edge computing includes:

[0009] The edge side uses a deep learning model based on a non-learning layer to train the collected video surveillance data on a video analysis model to obtain model parameters, and distributes the model parameters to the terminal devices;

[0010] Based on the model parameters distributed by the edge side, the terminal device uses a deep learning model of the non-learning layer to construct a video inference model at the terminal, and uses the video inference model to perform real-time analysis on the collected video surveillance data, and uploads the video surveillance data and the analysis results to the edge side.

[0011] Preferably, the method further includes: the cloud integrates the video analysis models of each edge side and distributes the video analysis model parameters to each edge side.

[0012] Preferably, the number of parameters to be learned by the deep learning model based on the non-learning layer is affected by the fully connected layer in the feature fusion part. During the feature extraction stage, the convolutional kernels that need to be learned through training in the convolution are replaced with fixed or randomly generated binary convolutional kernels. During the feature fusion stage, a learnable fully connected network structure is used, and during the training process, weighted calculation is performed on the video surveillance data features extracted by the neural network model.

[0013] Preferably, when the deep learning model based on the non-learning layer randomly generates convolutional kernels, seed is used to control the random process. When transmitting the model parameters, the value of seed is increased on the basis of the fully connected layer parameters. On the terminal device, the same model structure in the edge side model is reproduced through seed, and the surveillance video is inferred and analyzed;

[0014] The deep learning model structure based on the non-learning layer generates convolutional kernels of the non-learning layer through seed, extracts features in the target data through the non-learning layer, and transforms the features into a reasonable numerical range through an activation function, and fuses the learned features through a fully connected layer to form an analysis result of the deep learning model based on the non-learning layer.

[0015] As can be seen from the technical solutions provided by the embodiments of the present invention described above, the present invention proposes to improve the model in a non-learning layer manner in video analysis tasks based on edge computing and deep learning, optimize data transmission between the edge side and terminal devices, reduce communication latency, and at the same time maintain high-precision real-time video analysis capabilities.

[0016] Additional aspects and advantages of the present invention will be given in part in the following description, and these will become apparent from the following description, or can be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0018] Figure 1 It is a schematic diagram of the implementation principle of a neural network model training method based on edge computing provided by an embodiment of the present invention;

[0019] Figure 2 It is a schematic diagram of the basic functions of an edge side provided by an embodiment of the present invention;

[0020] Figure 3 It is a schematic diagram of the basic functions of a terminal device provided by an embodiment of the present invention;

[0021] Figure 4 It is a schematic diagram of the calculation process of a deep learning model based on a non-learning layer provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] The following details the embodiments of the present invention. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention and should not be construed as limiting the present invention.

[0023] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The phrase "and / or" used herein includes any unit and all combinations of one or more of the associated listed items.

[0024] Those skilled in the art can understand that, unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless defined as herein.

[0025] For the convenience of understanding the embodiments of the present invention, the following will further explain and illustrate with several specific embodiments in conjunction with the drawings, and each embodiment does not constitute a limitation to the embodiments of the present invention.

[0026] In recent years' research, it has been found that not all the parameters of the convolutional kernels in the model need to be adjusted through training, and sufficient accuracy can also be achieved through a partial generation method. Based on these studies, the embodiments of the present invention propose a new cloud / edge-end collaboration mode based on Non-learnable layer, which significantly reduces the amount of model parameter exchange between the cloud / edge-end while ensuring performance.

[0027] The embodiments of the present invention are carried out under the framework of edge computing. Edge computing is divided into a three-layer computing structure of "cloud - edge-end - terminal device". Among them, the cloud is responsible for integrating the video analysis models of each edge-end and sending the updated model parameters to each edge-end. The computing resources and tasks of the edge-end are relatively concentrated; the edge-end trains on the video analysis model based on the video monitoring data fed back by the terminal device to obtain model parameters, and sends the model parameters to the terminal device, and the terminal device uses the video inference model to perform real-time and accurate inference on the monitored video.

[0028] The implementation principle diagram of a neural network model training method based on edge computing provided by the embodiments of the present invention is as Figure 1As shown in the figure, the present invention mainly relates to a computing model between the edge side and the terminal device. The edge side uses a PC with sufficient computing resources, and the terminal device is a real-time video monitoring device based on Raspberry Pi. The edge side and the terminal device communicate through a wireless network to transfer data and parameters.

[0029] The method of the present invention mainly includes: the model training process on the edge side and the data collection and inference process on the terminal.

[0030] 1. Model training process on the edge side

[0031] In the architecture of edge computing, the edge side is a middleware with certain computing resources. In the present invention, the edge side uses a PC with sufficient computing resources, and its integrity in computer structure and flexibility in replacing computing resources can effectively handle complex tasks such as video processing and analysis. Figure 2 It is a schematic diagram of the basic functions of an edge side provided by an embodiment of the present invention. The edge side realizes the functions of data reception, model training, and parameter distribution.

[0032] Data reception: The edge side is far from the actual monitoring scenario and needs to collect and integrate the video monitoring data collected by the terminal monitoring device for unified processing and training of the data.

[0033] Model training: The edge side can utilize computing resources such as GPUs to train the collected video monitoring data using a deep learning model based on non-learning layers, and obtain faster convergence and higher accuracy by taking advantage of resource advantages.

[0034] Parameter distribution: After sufficient training of the deep learning model, model parameters are obtained, and the edge side distributes the model parameters to the terminal device, endowing the terminal device with the ability to perform fast and accurate video inference.

[0035] 2. Data collection and inference process on the terminal

[0036] Terminal devices often have a lightweight computing structure. In the present invention, the terminal uses a Raspberry Pi equipped with a video monitoring module, which is in direct contact with the actual monitoring scenario and can achieve timely information collection and feedback.

[0037] The terminal device realizes the functions of data collection and upload and real-time analysis and inference.

[0038] Data collection and upload: The terminal device collects video monitoring data in a specific scenario through the video monitoring module. Due to limited storage resources, the terminal needs to timely upload the collected video monitoring data to the edge side or cloud with sufficient resources for upper-layer structure training and information archiving.

[0039] Real-time analysis and inference: Based on the model parameters distributed at the edge, the deep learning model terminal using the non-learning layer can directly construct a lightweight video inference model adapted to the terminal computing resources, perform real-time analysis on the surveillance video, and execute common vision-based deep learning tasks such as target recognition according to specific scenarios.

[0040] II. Deep learning model based on non-learning layer

[0041] Existing model compression methods mainly reduce the computational complexity of the model from aspects such as model structure and nodes to achieve the purpose of reducing training time and communication latency. Generally speaking, due to the restriction of task complexity, these methods still need to balance between the scale and accuracy of the model. When the lower limit of the model scale is relatively high, the latency problem still cannot be well solved. Therefore, the present invention proposes a deep learning model based on the non-learning layer, which can keep the number of parameters to be trained and transmitted over the network small when the model is large, and improve the efficiency of real-time video analysis.

[0042] Common deep learning models used in video analysis need to use convolutional structures to analyze and extract target objects in data. In the context of edge computing, common neural network models generally include lightweight deep learning models such as VGG, MobileNet, ShuffleNet, and EfficientNet. Although convolution has achieved a more lightweight goal in terms of the number of parameters compared to fully connected deep learning models by using weight sharing, as the depth and width of the model increase, the number of parameters and computational complexity also increase to varying degrees.

[0043] Figure 4 Schematic diagram of the calculation process of a deep learning model based on the non-learning layer provided by an embodiment of the present invention. As Figure 4 shown, according to the tasks to be executed, the deep learning model structure can be divided into three stages. First, generate the convolution kernels of the non-learning layer through seeds, then extract the features in the target data through the non-learning layer, and transform the features into a reasonable numerical range through the activation function. Finally, fuse the learned features through the fully connected layer to form the analysis result of the deep learning model based on the non-learning layer.

[0044] The basic idea of the deep learning model of the non-learning layer is that in the feature extraction stage, the convolutional kernels that need to be learned through training in convolution are replaced with fixed or randomly generated binary convolutional kernels. In the feature fusion stage, network structures such as learnable fully connected layers are used, and reasonable weighted calculations are performed on the video surveillance data features extracted by the neural network model during the training process. Since the outputs of the original convolution and the convolution of the non-learning layer are mapped to the same interval under the action of the activation function, theoretically, through the weighted output of the fully connected layer, the improved model can definitely achieve the same monitoring video analysis results as the original model.

[0045] Since the convolutional kernels of the deep learning model of the non-learning layer do not need to be adjusted during the training process, no matter what form of adjustment is made to the neural network model in terms of depth, width, and model structure, it will not affect the dynamically adjusted number of parameters in the model. The number of parameters that need to be learned is only affected by the fully connected layer in the feature fusion part.

[0046] The above-mentioned deep learning model based on the non-learning layer can be fully deployed on edge devices and end devices. The convolutional kernel is crucial for the quality of the extracted features, and the convolutional kernel of the non-learning layer during training on the edge is randomly generated. In order to maintain the same accuracy as the edge when deploying on the terminal, the present invention proposes to use the seed to control the random process when randomly generating the convolutional kernel. When transmitting the model parameters, only the value of the seed needs to be added on the basis of the fully connected layer parameters. On the terminal device, the same model structure as in the edge model can be reproduced through the seed, and real-time and accurate inference analysis can be performed on the monitoring video.

[0047] In summary, the embodiments of the present invention propose to improve the model in the form of a non-learning layer in video analysis tasks based on edge computing and deep learning, optimize the data transmission between the edge device and the end device, reduce the communication delay, and at the same time maintain the high-precision real-time video analysis ability.

[0048] The embodiments of the present invention use a deep learning model based on the non-learning layer to reduce the amount of data transmitted between the cloud / edge and the end, and reduce the delay caused by network transmission.

[0049] The present invention uses the method of randomly generating convolutional kernels based on the seed, which effectively reduces the possible exposure of model parameter information, making it difficult for external attackers to obtain the gradient of the adversarial objective function for the sample data feature values, reducing the generation of adversarial samples, and ensuring the security of the video analysis system.

[0050] Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0051] As can be seen from the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0052] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, reference can be made to the partial description of the method embodiments. The device and system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0053] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A neural network model training method based on edge computing, characterized in that, Including: The edge side uses a deep learning model based on a non-learning layer to train the collected video surveillance data on the video analysis model, obtains model parameters, and distributes the model parameters to the terminal device. Based on the model parameters distributed by the edge side, the terminal device uses a deep learning model terminal based on a non-learning layer to construct a video inference model, uses the video inference model to perform real-time analysis on the collected video surveillance data, and uploads the video surveillance data and analysis results to the edge side. The cloud integrates the video analysis models of each edge side and distributes the video analysis model parameters to each edge side. The number of parameters to be learned in the deep learning model based on the non-learning layer is affected by the fully connected layer in the feature fusion part. During the feature extraction stage, the convolutional kernels that need to be learned through training in the convolution are replaced with fixed or randomly generated binary convolutional kernels. During the feature fusion stage, a learnable fully connected network structure is used, and the features of the video surveillance data extracted by the neural network model are weighted and calculated during the training process. When the deep learning model based on the non-learning layer randomly generates convolutional kernels, it uses seed to control the random process. When transmitting the model parameters, the value of seed is increased on the basis of the fully connected layer parameters. On the terminal device, the same model structure in the edge side model is reproduced through seed, and the surveillance video is inferred and analyzed. The deep learning model structure based on the non-learning layer generates the convolutional kernels of the non-learning layer through seed, extracts the features in the target data through the non-learning layer, and transforms the features into a reasonable numerical range through the activation function. The learned features are fused through the fully connected layer to form the analysis result of the deep learning model based on the non-learning layer. The deep learning model based on the non-learning layer is deployed on the edge side and the terminal device. The non-learning layer convolutional kernels during training on the edge side are randomly generated. When randomly generating convolutional kernels, seed is used to control the random process. When transmitting the model parameters, the value of seed is increased on the basis of the fully connected layer parameters. On the terminal device, the same model structure in the edge side model is reproduced through seed, and the surveillance video is inferred and analyzed.

Images