Iot cloud edge collaboration method and system

Abstract

The application discloses an Internet of Things cloud edge coordination method, which comprises the following steps: an Internet of Things cloud server determines an AI model to be used, parameters to be optimized and input data features according to a calculation task; the Internet of Things cloud server selects a corresponding Internet of Things edge terminal group according to the input data features, divides the Internet of Things edge terminal group into m sub-edge terminal groups according to the parameters to be optimized, and respectively distributes the calculation task to the corresponding sub-edge terminal groups; the Internet of Things cloud server synchronizes the AI model to each sub-edge terminal group, and iteratively updates the parameters to be optimized of the AI model in the sub-edge terminal group by using an artificial bee colony algorithm to obtain a local optimal solution of the AI model; the Internet of Things cloud server receives the local optimal solutions from each sub-edge terminal group, selects one local optimal solution as a global optimal solution from the local optimal solutions, and thus obtains optimal parameters. The application also discloses an Internet of Things cloud edge coordination system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet of Things, and more particularly to an Internet of Things cloud-edge coordination method and system. BACKGROUND

[0002] In the Internet of Things environment, there are a large number of Internet of Things edge computing devices, and a large amount of local data exists in these devices. At present, the local data of the edge computing devices is mostly transmitted to the cloud computing center for calculation to complete the calculation task, resulting in the problem of waste of operation resources of the Internet of Things edge computing devices. Therefore, how to complete the calculation task without uploading data and how to fully utilize the operation resources of the large number of edge computing devices have become urgent problems to be solved. SUMMARY

[0003] The present application solves the technical problem of providing an Internet of Things cloud-edge coordination method and system to fully utilize the operation resources of the large number of edge terminals of the Internet of Things and achieve the purpose of fast convergence of AI calculation tasks.

[0004] In a first aspect, the present application provides an Internet of Things cloud-edge coordination method, which comprises:

[0005] The Internet of Things cloud server determines the AI model to be used, the parameters to be optimized, and the input data features according to the calculation task;

[0006] The Internet of Things cloud server selects the corresponding Internet of Things edge terminal group according to the input data features, divides the Internet of Things edge terminal group into m sub-edge terminal groups according to the parameters to be optimized, and distributes the calculation task to the corresponding sub-edge terminal group respectively; wherein m is a positive integer;

[0007] The Internet of Things cloud server synchronizes the AI model to each sub-edge terminal group, and uses the artificial bee colony algorithm to iteratively update the parameters to be optimized of the AI model in each sub-edge terminal group to obtain the local optimal solution of the AI model;

[0008] The Internet of Things cloud server receives the local optimal solution from each sub-edge terminal group, and selects one local optimal solution as the global optimal solution from them, thereby obtaining the optimal parameters.

[0009] In a second aspect, the present application further provides an Internet of Things cloud-edge coordination system, which comprises an Internet of Things cloud server and a plurality of edge terminals. The Internet of Things cloud server comprises a center calculation module and a center scheduling module, and the edge terminals comprise an edge calculation module and an edge storage module, wherein,

[0010] The center computing module is used for determining an AI model to be used, parameters to be optimized and input data features according to a computing task, and selecting a corresponding Internet of Things edge terminal group according to the input data features, wherein the Internet of Things edge terminal group is composed of a plurality of edge terminals;

[0011] The center scheduling module is used for dividing the Internet of Things edge terminal group into m sub-edge terminal groups according to the parameters to be optimized, and distributing the computing task into the corresponding sub-edge terminal groups respectively, wherein m is a positive integer; and synchronizing the AI model into the edge terminals of each sub-edge terminal group, determining the number of leader bees, follower bees and honey sources, the maximum iteration number and the number of stagnation according to the artificial bee colony algorithm, determining the model error of the AI model as an adaptive value, and assigning roles to the edge terminals; wherein the roles are leader bees, follower bees or scout bees.

[0012] The edge storage module is used for storing locally collected data.

[0013] The edge computing module is used for iteratively updating the parameters to be optimized of the AI model according to the assigned roles, the locally collected data and the maximum iteration number, to obtain a local optimal solution of the AI model, and sending the local optimal solution to the center computing module.

[0014] The center computing module is further used for selecting a local optimal solution from the received local optimal solutions as a global optimal solution, to obtain an optimal parameter.

[0015] Compared with the prior art, in the present application, the Internet of Things cloud server can determine an AI model to be used, parameters to be optimized and input data features according to a computing task, to select a corresponding Internet of Things edge terminal group, and divide the Internet of Things edge terminal group into m sub-edge terminal groups, and distribute the computing task into the corresponding sub-edge terminal groups respectively, and can also synchronize the determined AI model to each sub-edge terminal group, and iteratively update the parameters to be optimized of the AI model in each sub-edge terminal group using the artificial bee colony algorithm, to obtain a local optimal solution of the AI model, and the Internet of Things cloud server can also select a local optimal solution from the received local optimal solutions from each sub-edge terminal group as a global optimal solution, to obtain an optimal parameter. It can be seen that the present application makes full use of the computing resources of the massive edge terminals of the Internet of Things, that is, the computing task can be decomposed as much as possible in the case that the edge terminals of the Internet of Things do not upload data, combined with the artificial bee colony algorithm, through the group intelligence mode, the cloud edge end cooperation is completed, the parallel computing of the task is realized, the rapid convergence of the AI computing task is realized, and the safety and privacy of the data are also guaranteed. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of an Internet of Things cloud edge cooperation method provided by the embodiments of the present application is shown.

[0017] Figure 2 A sub-flow schematic diagram of the Internet of Things cloud-edge collaboration method provided by the embodiment of the present application.

[0018] Figure 3 A schematic block diagram of the Internet of Things cloud-edge collaboration system provided by the embodiment of the present application. DETAILED DESCRIPTION

[0019] In order for those skilled in the art to more clearly understand the purposes, technical solutions and advantages of the present application, the present application will be further described below in conjunction with the drawings and embodiments.

[0020] Reference Figure 1 , Figure 1 A flowchart of the Internet of Things cloud-edge collaboration method provided by the embodiment of the present application. As shown in the figure, the Internet of Things cloud-edge collaboration method comprises the following steps:

[0021] S101, the Internet of Things cloud server determines the AI model to be used, the parameters to be optimized and the input data features according to the computing task.

[0022] In the present application, the computing task is an artificial intelligence model training computing task; the AI model can be a BP neural network, an RBF (Radial Basis Function) neural network, a perceptron neural network, a linear neural network, a self-organizing neural network or a feedback neural network, or can be a linear model, a support vector machine model, a decision tree model or a naive Bayes model; the input data features can be image data features, sound data features, or custom features according to specific task text data, etc.

[0023] In the present embodiment, the parameters to be optimized can be the weights Wi of the input layer and the hidden layer of the neural network and the weights Wj of the hidden layer and the connection layer, or other parameters in the machine learning algorithm model.

[0024] S102, the Internet of Things cloud server selects a corresponding Internet of Things edge terminal group according to the input data features, and divides the Internet of Things edge terminal group into m sub-edge terminal groups according to the parameters to be optimized, and respectively distributes the computing task to the corresponding sub-edge terminal group.

[0025] In the present application, the purpose of the Internet of Things edge terminal is known in advance, so the data features in the Internet of Things edge terminal are known, and the input data features of the computing task can be selected to determine the appropriate Internet of Things edge terminal as the Internet of Things edge terminal group T.

[0026] In the step, the Internet of Things edge terminal group is divided into m sub-edge terminal groups according to the parameter to be optimized, specifically including: dividing the value of the parameter to be optimized into n quadrants; in the embodiment, the value range of the parameter can be estimated by the experience in the early stage, and the value range is divided into n different quadrants; the Internet of Things edge terminal group is divided into m sub-edge terminal groups according to the n quadrants; wherein n and m are positive integers, and m=n. That is, the value of the parameter to be optimized is divided into A1, A2, A3, …, An, a total of n quadrants, and then the Internet of Things edge terminal group T is further divided into T1, T2, T3, …, Tm, a total of m sub-edge terminal groups, that is, one quadrant corresponds to one sub-edge terminal group.

[0027] In S103, the Internet of Things cloud server synchronizes the AI model to each sub-edge terminal group, and uses the artificial bee colony algorithm to iteratively update the parameter to be optimized of the AI model in each sub-edge terminal group to obtain the local optimal solution of the AI model.

[0028] In the application, the AI model synchronized to each sub-edge terminal group is an AI model that has been initialized, so in the step, the Internet of Things cloud server controls m sub-edge terminal groups, and synchronizes the AI model that has been initialized and determined according to the computing task to the m sub-edge terminal groups, and then uses the artificial bee colony algorithm to iteratively update the parameter of the AI model in each sub-edge terminal group.

[0029] Specifically, as shown in Figure 2 the step of using the artificial bee colony algorithm to iteratively update the parameter to be optimized of the AI model in each sub-edge terminal group to obtain the local optimal solution of the AI model, specifically including:

[0030] In S1031, the Internet of Things cloud server determines the number of leading bees, following bees and honey sources, the maximum number of iterations and the number of stagnations according to the artificial bee colony algorithm, and determines the model error of the AI model as the fitness value.

[0031] In the step, the model error is mean square error MSE.

[0032] In S1032, the Internet of Things cloud server determines some edge terminals in each sub-edge terminal group as leading bees.

[0033] In S1033, the edge terminal as the leading bee updates the AI model synchronized to the edge terminal according to the value of the parameter to be optimized, randomly extracts data stored in a predetermined number of edge terminals and inputs the data into the AI model, calculates the model error, and transmits the model error to the Internet of Things cloud server, and if the model error is better than the old fitness value, the model error is updated.

[0034] S1034, The Internet of Things cloud server determines at least one edge terminal in each sub-edge terminal group as a follower bee, each follower bee updates the AI model synchronized to the edge terminal according to the value of the parameter to be optimized, and randomly extracts data stored in a preset number of edge terminals to input the AI model, calculates the model error, and transmits the model error to the Internet of Things cloud server. If the model error is better than the old adaptive value, it is updated to obtain a local optimal solution.

[0035] In this step, the Internet of Things cloud server can determine the follower bee in the sub-edge terminal group through the roulette algorithm.

[0036] S1035, If the edge terminal as a follower bee does not obtain a model error better than the old adaptive value, discard the model error.

[0037] In the present application, if the edge terminal as a follower bee does not obtain a model error better than the old adaptive value within a preset number of iterations, discard the corresponding honey source, and determine the scout bee to find a better honey source by the Internet of Things cloud server.

[0038] S1036, The Internet of Things cloud server determines at least one scout bee in the corresponding sub-edge terminal group to update the AI model synchronized to the edge terminal according to the value of the parameter to be optimized, and randomly extracts a preset number of data stored in the edge terminal to input the AI model, calculates a new model error, and transmits the model error to the Internet of Things cloud server. If the model error is better than the old adaptive value, it is updated.

[0039] S1037, continuously iteratively update until the maximum number of iterations, the adaptive value reaches convergence, and a local optimal solution of the AI model is obtained.

[0040] In the above steps, the edge terminal updates the parameters of the AI model in the parameter quadrant interval according to the role (leader bee, follower bee or scout bee) assigned by the Internet of Things cloud server, and after the parameter update, the local data stored in the edge terminal is randomly extracted to input the AI model to calculate the adaptive value of the model error. After multiple iterations of updating, convergence is achieved in m different sub-edge terminal groups, and the Internet of Things cloud server selects the best parameters by comparing the adaptive values in different sub-edge terminal groups.

[0041] S104, The Internet of Things cloud server receives a local optimal solution from each sub-edge terminal group, and selects a local optimal solution as a global optimal solution, thereby obtaining an optimal parameter.

[0042] This step specifically includes: the Internet of Things cloud server receives a local optimal solution from each sub-edge terminal group, compares the sizes of the received local optimal solutions, selects the local optimal solution with the smallest value as the global optimal solution, and thereby obtains an optimal parameter.

[0043] In conclusion, the Internet of Things cloud edge coordination method utilizes the bee colony algorithm, can realize cloud edge end coordination and rapid convergence of the model, fully utilizes the operation resources of the massive edge terminals of the Internet of Things, fully utilizes the data of the edge side terminals of the Internet of Things for calculation, guarantees the safety and privacy of the data, can also avoid the training calculation from falling into a local optimal value, and divides the parameter value into multiple quadrants, through the multiple quadrant bee colony algorithm, can decompose the calculation task as much as possible, realizes parallel calculation of the task, and further reduces the calculation time.

[0044] Referring to Figure 3 , Figure 3 A schematic block diagram of an Internet of Things cloud edge coordination system provided for an embodiment of the present application is shown in the figure. As shown in the figure, the Internet of Things cloud edge coordination system comprises an Internet of Things cloud server 11 and a plurality of edge terminals 12, the Internet of Things cloud server 11 comprises a center calculation module 112 and a center scheduling module 111, and the edge terminal 12 comprises an edge calculation module 121 and an edge storage module 122.

[0045] The center computing module 112 is configured to determine an AI model to be used, parameters to be optimized, and input data features according to a computing task, and select a corresponding Internet of Things edge terminal group according to the input data features, the Internet of Things edge terminal group being composed of a plurality of edge terminals. Specifically, in this embodiment, the computing task is an artificial intelligence model training computing task; the AI model can be a BP neural network, an RBF (Radial Basis Function) neural network, a perceptron neural network, a linear neural network, a self-organizing neural network, or a feedback neural network, or can be a linear model, a support vector machine model, a decision tree model, or a Naive Bayes model; the input data features can be image data features, sound data features, or text data features customized according to a specific task; and the parameters to be optimized can include weights Wi of input layers and hidden layers of a neural network and weights Wj of hidden layers and connection layers, or other parameters of a machine learning algorithm model. The center scheduling module 111 is configured to divide the Internet of Things edge terminal group composed of a plurality of edge terminals 12 into m sub-edge terminal groups according to the parameters to be optimized, and distribute the computing task to the corresponding sub-edge terminal groups respectively. Specifically, the sub-edge terminal group is also composed of a plurality of edge terminals 12, the center scheduling module 111 determines the required Internet of Things edge terminal group T according to the determined input data features, and divides the values of the parameters to be optimized into n quadrants; the Internet of Things edge terminal group is divided into m sub-edge terminal groups according to the n quadrants; wherein n and m are positive integers, and m = n; and the center scheduling module 111 also synchronizes the AI model to the edge terminals 12 of each sub-edge terminal group, determines the number of leader bees, follower bees, and honey sources, the maximum number of iterations, and the number of stalls according to the artificial bee colony algorithm, determines the model error of the AI model as the fitness value, and allocates roles to the edge terminals 12; wherein the roles are leader bees, follower bees, or scout bees. The edge storage module 122 is configured to store locally collected data. The edge computing module 121 iteratively updates the parameters to be optimized of the AI model according to the allocated role, the locally collected data, and the maximum number of iterations, to obtain a local optimal solution of the AI model, and sends the local optimal solution to the center computing module 112. The center computing module 112 is further configured to select a local optimal solution from the received local optimal solutions from each sub-edge terminal group as a global optimal solution, thereby obtaining an optimal parameter, that is, comparing the sizes of the received local optimal solutions from the edge computing module 121, selecting the local optimal solution with the smallest value as the global optimal solution, and thereby obtaining the optimal parameter.

[0046] Preferably, the edge computing module 121 is specifically configured to: if the edge terminal 12 is assigned as a leading bee, the edge computing module 121 is configured to update the AI model synchronized to the edge terminal 12 according to the value of the parameter to be optimized, randomly extract a preset number of local data stored in the edge storage module 122 into the AI model, calculate the model error, and transmit the model error to the central computing module 112, and continuously iterate the calculation until the maximum iteration number, to obtain a local optimal solution of the AI model; the central scheduling module 111 determines the follower bee in the sub-edge terminal group through the roulette algorithm, if the edge terminal 12 is assigned as a follower bee, the edge computing module 121 is configured to update the AI model synchronized to the edge terminal 12 according to the value of the parameter to be optimized, randomly extract a preset number of local data stored in the edge storage module 122 into the AI model, calculate the model error, and transmit the model error to the central computing module 112, and continuously iterate the calculation until the maximum iteration number, to obtain a local optimal solution of the AI model.

[0047] Further, if the edge computing module 121 as a follower bee does not obtain a model error better than the old adaptive value within a preset iteration number, the corresponding honey source is discarded, and a scout bee is determined by the central scheduling module 111 to find a better honey source; the edge computing module 121 is further configured to: if the edge terminal 12 is assigned as a scout bee, the edge computing module 121 is configured to update the AI model synchronized to the edge terminal 12 according to the value of the parameter to be optimized, randomly extract a preset number of local data stored in the edge storage module 122 into the AI model, calculate the model error, and transmit the model error to the central computing module 112, and continuously iterate the calculation until the maximum iteration number, to obtain a local optimal solution of the AI model.

[0048] Understandably, the edge computing module 121 in the edge terminal updates the parameters of the AI model in the parameter quadrant interval according to the role (leading bee, follower bee or scout bee) assigned by the central scheduling module 111, after the parameter update, the local data stored in the edge storage module 122 is randomly extracted into the AI model to calculate the adaptive value of the model error, after multiple iteration updates, convergence is achieved in m different sub-edge terminal groups, and the central computing module 112 can select the best parameters by comparing the adaptive values in different sub-edge terminal groups.

[0049] It should be noted that the descriptions of each embodiment in the above embodiments have different emphases. Parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. For the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Those skilled in the art can make various equivalent changes and improvements based on the above embodiments, and all equivalent variations or modifications made within the scope of the claims should fall within the protection scope of the present invention.

Claims

1. An IoT cloud-edge collaboration method, characterized in that, include: The IoT cloud server determines the AI ​​model to be used, the parameters to be optimized, and the characteristics of the input data based on the computing task. The IoT cloud server selects and determines the corresponding IoT edge terminal group based on the characteristics of the input data, and divides the IoT edge terminal group into m sub-edge terminal groups according to the parameters to be optimized, and distributes the computing tasks to the corresponding sub-edge terminal groups respectively; where m is a positive integer; The IoT cloud server synchronizes the AI ​​model to each sub-edge terminal group and uses the artificial bee colony algorithm to iteratively update the parameters to be optimized in the AI ​​model in each sub-edge terminal group in order to obtain the local optimal solution of the AI ​​model. The IoT cloud server receives local optimal solutions from each sub-edge terminal group and selects one of the local optimal solutions as the global optimal solution to obtain the optimal parameters. Specifically, the step of using the artificial bee colony algorithm to iteratively update the parameters to be optimized in the AI ​​model of each sub-edge terminal group to obtain the local optimum of the AI ​​model includes: The IoT cloud server determines the number of leader bees, follower bees, and nectar sources, the maximum number of iterations, and the number of dwell times based on the artificial bee colony algorithm, and determines the model error of the AI ​​model as the adaptation value. The IoT cloud server identifies certain edge terminals within each sub-edge terminal group as leader bees; As the leading edge terminal, the AI ​​model of the edge terminal is updated and synchronized according to the value of the parameter to be optimized. A preset number of data stored in the edge terminal are randomly selected and input into the AI ​​model to calculate the model error. The model error is then transmitted to the IoT cloud server. If the model error is better than the old adaptive value, it is updated. The IoT cloud server identifies at least one edge terminal in each sub-edge terminal group as a follower bee. Each follower bee updates and synchronizes the AI ​​model of the edge terminal according to the value of the parameter to be optimized, and randomly selects a preset number of data stored in the edge terminal to input into the AI ​​model, calculates the model error, and transmits the model error to the IoT cloud server. If the model error is better than the old fitness value, it is updated to obtain a local optimum.

2. The IoT cloud-edge collaboration method as described in claim 1, characterized in that, The step of using the artificial bee colony algorithm to iteratively update the parameters to be optimized for the AI ​​model in each sub-edge terminal group to obtain the local optimum of the AI ​​model also includes: If a model error better than the old fitness value is not obtained at the edge terminal of the follower bee, then the model error is discarded. The IoT cloud server identifies at least one scout bee in the corresponding sub-edge terminal group to update the AI ​​model synchronized to the edge terminal according to the value of the parameter to be optimized, and randomly selects a preset number of data stored in the edge terminal to input into the AI ​​model, calculates the new model error, and transmits the model error to the IoT cloud server. If the model error is better than the old fitness value, it is updated. The algorithm iterates and updates continuously until the maximum number of iterations is reached, at which point the fitness value converges, and a local optimum of the AI ​​model is obtained.

3. The IoT cloud-edge collaboration method as described in claim 1, characterized in that, The step of dividing the IoT edge terminal group into m sub-edge terminal groups according to the parameters to be optimized specifically includes: The parameters to be optimized are divided into n quadrants; where n is a positive integer. The IoT edge terminal group is divided into m sub-edge terminal groups based on n quadrants.

4. The IoT cloud-edge collaboration method as described in claim 1, characterized in that, The IoT cloud server receives locally optimal solutions from each sub-edge terminal group and selects one locally optimal solution as the global optimal solution to obtain the optimal parameters, specifically including: The IoT cloud server receives local optimal solutions from each sub-edge terminal group, compares the size of the received local optimal solutions, and selects the local optimal solution with the smallest value as the global optimal solution, thereby obtaining the optimal parameters.

5. The IoT cloud-edge collaboration method as described in claim 1, characterized in that, The parameters to be optimized include the weights Wi of the input and hidden layers of the neural network, and the weights Wj of the hidden and connected layers.

6. An IoT cloud-edge collaborative system, characterized in that, It includes an IoT cloud server and multiple edge terminals. The IoT cloud server includes a central computing module and a central scheduling module, and the edge terminals include an edge computing module and an edge storage module. The central computing module is used to determine the AI ​​model to be used, the parameters to be optimized, and the input data features according to the computing task, and to select and determine the corresponding IoT edge terminal group according to the input data features. The IoT edge terminal group consists of multiple edge terminals. The central scheduling module is used to divide the IoT edge terminal group into m sub-edge terminal groups according to the parameters to be optimized, and distribute the computing tasks to the corresponding sub-edge terminal groups, where m is a positive integer; and synchronize the AI ​​model to the edge terminals of each sub-edge terminal group, determine the number of leader bees, follower bees, and nectar sources, the maximum number of iterations, and the number of dwell times according to the artificial bee colony algorithm, determine the model error of the AI ​​model as the fitness value, and assign roles to the edge terminals; wherein, the roles are leader bees, follower bees, or scout bees; The edge storage module is used to store locally collected data; The edge computing module iteratively updates the parameters of the AI ​​model to be optimized based on the assigned role, locally collected data, and maximum number of iterations to obtain the local optimum of the AI ​​model and send it to the central computing module. The central computing module is also used to select a local optimal solution from the received local optimal solutions as the global optimal solution, thereby obtaining the optimal parameters.

7. The IoT cloud-edge collaborative system as described in claim 6, characterized in that, The edge computing module is specifically used for: If the edge terminal is assigned as the leader bee, the edge computing module is used to update and synchronize the AI ​​model of the edge terminal according to the value of the parameter to be optimized, and randomly extract a preset amount of local data from the edge storage module to input into the AI ​​model, calculate the model error, and transmit the model error to the central computing module, and continuously iterate the calculation until the maximum number of iterations is reached to obtain the local optimal solution of the AI ​​model. If the edge terminal is assigned as a follower bee, the edge computing module updates and synchronizes the AI ​​model of the edge terminal according to the value of the parameter to be optimized, randomly extracts a preset amount of local data from the edge storage module and inputs it into the AI ​​model, calculates the model error, transmits the model error to the central computing module, and continuously iterates the calculation until the maximum number of iterations is reached to obtain the local optimum solution of the AI ​​model.

8. The IoT cloud-edge collaborative system as described in claim 7, characterized in that, The edge computing module is also used to: when the edge terminal is assigned as a reconnaissance bee, the edge computing module updates and synchronizes the AI ​​model of the edge terminal according to the value of the parameter to be optimized, randomly extracts a preset number of local data from the edge storage module and inputs them into the AI ​​model, calculates the model error, transmits the model error to the central computing module, and continuously iterates the calculation until the maximum number of iterations is reached to obtain the local optimal solution of the AI ​​model.

9. The IoT cloud-edge collaborative system as described in claim 6, characterized in that, The parameters to be optimized include the weights Wi of the input and hidden layers of the neural network, and the weights Wj of the hidden and connected layers.

Images