Neural network parameter processing method, device and equipment

Abstract

The present application provides a neural network parameter processing method, device and equipment, which relates to the field of machine learning technology. The method includes: determining the parameters of the target population according to the pre-stored neural network; dividing the target population into multiple sub-populations according to a preset position update method; initializing the particles in each sub-population and updating the initial vector of each particle; inputting the vector to be input to each particle, updating the particle position according to the particle position update method, and obtaining a new vector and fitness function value; updating the historical optimal solution of each particle in each particle population and the optimal solution of the sub-population of the particle population, and updating the temporary global optimal solution of the target population; outputting the temporary global optimal solution after the last iteration as the target global optimal solution; determining the target weights and target thresholds required for the pre-stored neural network. The present application can save the time cost of obtaining a better initial weight of the neural network.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, in particular to the field of neural network technology, and in particular to a neural network parameter processing method, device and equipment. Background Art

[0002] Neural networks have strong learning, nonlinear modeling and parallel processing capabilities, and have performed well in various fields and achieved remarkable results, and have been widely used. However, since the training process of neural networks takes a lot of time, how to save the training time of neural networks has become a key issue in the industry.

[0003] At present, in the prior art, optimization algorithms such as genetic algorithms and ant colony algorithms are generally used to optimize the parameters of the neural network to calculate the initial weights in the parameters of the neural network and complete the optimization of the initial weights of the neural network parameters.

[0004] However, the inventors have found that the prior art has at least the following technical problems: the performance of the existing algorithm is limited, and the accuracy and efficiency of the initial weights obtained still have a lot of room for improvement. Therefore, there is an urgent need for a neural network parameter processing method to obtain better neural network initial weights with less time cost, thereby saving time cost for the neural network training process. Summary of the invention

[0005] The present application provides a neural network parameter processing method, device and equipment, which can obtain better neural network initial weights with less time cost, thereby saving time cost for the neural network training process.

[0006] In a first aspect, the present application provides a neural network parameter processing method, comprising:

[0007] Determine the parameters of the target population according to the pre-stored neural network;

[0008] Dividing the target population into multiple sub-populations according to a preset position update method, wherein each sub-population includes at least two particles, and the particle position update methods of different sub-populations are different;

[0009] Initializing the particles in each subpopulation, updating the initial vector of each particle, and determining the updated initial vector as the to-be-input vector of the particle, wherein the initial vector is determined according to the weights and thresholds included in the pre-stored neural network;

[0010] Inputting the vector to be input into each particle, updating the particle position of the particle according to the particle position update method corresponding to the particle, and obtaining a new vector and fitness function value;

[0011] According to the fitness function value corresponding to each particle in the target population, the historical optimal solution of each particle in each particle population and the sub-population optimal solution of the particle population are updated, and according to all the sub-population optimal solutions, the temporary global optimal solution of the target population is updated;

[0012] Guide the update of all particles through the temporary global optimal solution, use the corresponding new vector as the input vector for the next iteration, return to the step of inputting the input vector to each particle, update the particle position of the particle according to the particle position update method, and obtain a new vector and a fitness function value, until it is detected that the update iteration reaches the iteration stop condition, and output the temporary global optimal solution after the last iteration as the target global optimal solution, wherein the target global optimal solution is a target vector including the weights and thresholds required by the pre-stored neural network;

[0013] According to the target vector, the target weight and target threshold required by the pre-stored neural network are determined.

[0014] In a possible implementation, the preset position update method includes a change method of the inertia weight and the acceleration factor as the number of iterations increases; accordingly, the target population is divided into multiple sub-populations according to the preset position update method, wherein each sub-population includes at least two particles, including: selecting particles from the target population to form a first sub-population according to a change method in which the inertia weight and the acceleration factor are both constant values ​​as the number of iterations increases; selecting particles from the target population to form a second sub-population according to a change method in which the inertia weight is randomly updated and the acceleration factor is a constant value as the number of iterations increases; selecting particles from the target population to form a third sub-population according to a change method in which the inertia weight is updated with a first formula and the acceleration factor is updated with a second formula as the number of iterations increases.

[0015] In a possible implementation, the first formula is:

[0016] ω k =ω0·A(k)

[0017] In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population, A(k)=1-(1+e -k+N / 2 ) -1 , where e is a natural constant and N is the preset maximum number of iterations;

[0018] The second formula is:

[0019]

[0020] In the formula, c 1kis the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration.

[0021] In a possible implementation, the first formula is:

[0022] ω k =ω0·(0.99) k

[0023] In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population;

[0024] The second formula is:

[0025]

[0026] In the formula, c 1k is the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration.

[0027] In a possible implementation, the number and proportion of particles in the first subpopulation, the second subpopulation, and the third subpopulation are determined according to the complexity of the problem to be solved by the pre-stored neural network. In a possible implementation, the particles in each subpopulation are initialized and the initial vector of each particle is updated, including: using a random number generation function to generate an initialization random number corresponding to each particle in each subpopulation; determining the number of values ​​of the initial vector according to the initial vector of the particle; and updating the initial vector of the particle according to a preset value range, the number of values, and the initialization random number. In a possible implementation, before the particle position of the particle is updated according to the particle position update method corresponding to the particle, and a new vector is obtained, it also includes: when a particle is detected to belong to the second subpopulation, a random number generation function is used to generate a random number corresponding to the particle; according to the random number corresponding to the particle and a preset inertia weight range, the inertia weight of the particle is randomly updated, and the calculation formula is:

[0028] ω n =ω min +(ω max -ω min)·p

[0029] In the formula, ω n is the updated inertia weight, ω max is the maximum inertia weight in the preset inertia weight range, ω min is the minimum inertia weight in the preset inertia weight range, and p is a random number corresponding to the particle.

[0030] In a possible implementation, the method further includes: when a particle is detected to belong to a particle of the third sub-population, the inertia weight is divided into three change stages as the number of iterations increases; the three change stages are respectively a first stage in which the inertia weight in the early stage of the iteration is kept slightly less than a preset weight upper limit value, a second stage in which the inertia weight in the middle stage of the iteration gradually decreases as the iteration proceeds, and a third stage in which the inertia weight in the late stage of the iteration is kept slightly greater than a preset weight lower limit value.

[0031] In a second aspect, the present application provides a neural network parameter processing device, comprising:

[0032] A parameter definition module is used to determine the parameters of the target population based on the pre-stored neural network;

[0033] A population division module is used to divide the target population into multiple sub-populations according to a preset position update method, wherein each sub-population includes at least two particles, and the particle position update methods of different sub-populations are different;

[0034] An initialization module, used to initialize the particles in each sub-population and update the initial vector of each particle, wherein the initial vector is determined according to the weights and thresholds included in the pre-stored neural network;

[0035] The particle swarm iterative calculation module is used to determine the vector to be input according to the weight and threshold corresponding to each particle; input the vector to be input to each particle, update the particle position of the particle according to the particle position update method corresponding to the particle, and obtain a new vector and fitness function value;

[0036] The particle swarm iterative calculation module updates the historical optimal solution of each particle in each particle population and the sub-population optimal solution of the particle population according to the fitness function value corresponding to each particle in the target population, and updates the temporary global optimal solution of the target population according to all the sub-population optimal solutions;

[0037] The particle swarm iterative calculation module is further used to guide the update of all particles through the temporary global optimal solution, use the corresponding new vector as the vector to be input for the next iteration, return the step of inputting the vector to be input to each particle, updating the particle position of the particle according to the particle position update method, and obtaining a new vector and a fitness function value, until it is detected that the update iteration reaches the iteration stop condition, and output the temporary global optimal solution after the last iteration as the target global optimal solution, wherein the target global optimal solution is a target vector containing the weights and thresholds required by the pre-stored neural network;

[0038] The particle swarm iterative calculation module is also used to determine the target weight and target threshold required by the pre-stored neural network according to the target vector.

[0039] In a third aspect, the present application provides a neural network parameter processing device, including: at least one processor and a memory;

[0040] The memory stores computer-executable instructions;

[0041] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the neural network parameter processing method described in the first aspect above.

[0042] The neural network parameter processing method, device and equipment provided by the present application first determine the parameters of the target population according to the pre-stored neural network, and then divide the target population into multiple sub-populations with different particle position update methods, increase the diversity of the target population, realize information exchange in the process of generating the global optimal solution, and improve the search efficiency. In addition, an adaptive particle position update method is introduced in the iteration process to update the particle position, and by comparing the fitness function value of the particle, the historical optimal solution of each particle in each sub-population and the sub-population optimal solution of the sub-population are updated, and the temporary global optimal solution of the target population is updated. Finally, the target global optimal solution is obtained at the end of the iteration, and the target weights and target thresholds required for the pre-stored neural network are determined according to the target global optimal solution. In the process of solving the pre-stored neural network, the global optimization ability of the algorithm is improved, and then the overall accuracy of the algorithm is improved, and a better initial weight of the neural network can be obtained with less time cost, thereby saving time cost for the training process of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A schematic diagram of an application scenario of the neural network parameter processing method provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of a neural network parameter processing method provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of the structure of a neural network parameter processing device provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of the hardware structure of a neural network parameter processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] At present, in addition to the optimization algorithm in the background technology, the prior art has used the particle swarm algorithm to optimize the neural network. However, the inventors found in practice that the most primitive form of the particle swarm algorithm has little or no improvement on the intelligent optimization algorithm, resulting in the accuracy and efficiency of the algorithm need to be improved to solve the problem of low accuracy and efficiency of optimizing the neural network in the prior art.

[0050] In order to solve the above technical problems, the embodiments of the present application provide the following technical concepts for solving the problems: First, by dividing the particles in the entire particle group into three categories, the diversity of the population is increased, and information exchange is realized in the process of generating the global optimal solution, thereby improving the search efficiency. Among them, the first type of particles is suitable for solving low-complexity problems, the second type of particles is suitable for solving medium-complexity problems, and the third type of particles is suitable for solving high-complexity problems, thereby improving the overall efficiency of the algorithm. In addition, by introducing a method of adaptively adjusting the inertia weight and acceleration factor in the third type of particles, adaptive adjustment of the particle iteration process is achieved, and the intelligence level of the algorithm is improved to ensure that the algorithm can obtain a better global optimal solution and improve the accuracy of the algorithm.

[0051] Figure 1 A schematic diagram of an application scenario of the neural network parameter processing method provided in an embodiment of the present application, such as Figure 1 As shown, it includes: a terminal 101 and a server 102.

[0052] The terminal 101 is used to display the algorithm output results. The server 102 is used to receive vectors transmitted from the cloud, other servers or mobile terminals, and execute the steps of the neural network parameter processing method to generate output results and train the neural network.

[0053] Figure 2 This is a flow chart of the neural network parameter processing method provided in the embodiment of the present application. The execution subject of this embodiment can be Figure 1 The server 102 in the illustrated embodiment may also be other computer-related devices, and there is no particular limitation on this embodiment.

[0054] like Figure 2 As shown, the neural network parameter processing method includes:

[0055] S201: Determine the parameters of the target population according to the pre-stored neural network.

[0056] In this embodiment, the pre-stored neural network refers to a neural network that can be used to solve real-world related problems, for example: a neural network for inputting a feature vector obtained by a feature extraction process of a multi-beam seafloor sonar image. The parameters of the target population include constants and variable parameters of the adaptive particle swarm algorithm, where the constant parameter can be π, and the variable parameters can include the number of particles, the maximum number of iterations, the inertia weight, and the acceleration factor. For example: the number of particles is set to 100, and the maximum number of iterations is set to 100.

[0057] In an optional embodiment of the present application, the number of particles in the parameters of the target population can be increased accordingly as the complexity of the problem to be solved by the neural network to be optimized increases. For example, when the number of particles in the shortest path problem when searching for 500 targets on the seabed is calculated, the number of particles can be set to 50, and the number of particles in the shortest path problem when searching for 1000 targets is 100. It should be noted that the number of particles can increase as the complexity of the problem to be solved increases, and the increase described here is not necessarily a linearly related increase.

[0058] In an optional embodiment of the present application, the setting of the maximum number of iterations can be increased or decreased according to the iterative convergence effect during the iteration process of the particle swarm algorithm.

[0059] S202: Divide the target population into a plurality of sub-populations according to a preset position update method, wherein each sub-population includes at least two particles, and the particle position update methods of different sub-populations are different.

[0060] In this embodiment, the preset position update method can be a preset formula, corresponding relationship or mapping rule of particle position changing with the number of iterations. The position update method is different for different types of particles, and the number of sub-populations can be determined when determining the parameters of the target population.

[0061] Specifically, in an optional embodiment of the present application, the preset position update method includes a change method of the inertia weight and the acceleration factor as the number of iterations increases. Accordingly, step S202 specifically includes:

[0062] S202a: Select particles from the target population to form a first sub-population in a manner that the inertia weight and the acceleration factor change in a constant manner as the number of iterations increases.

[0063] In this embodiment, as the number of iterations increases, the inertia weight and the acceleration factor are both constant values, and the constant values ​​can be preset constant values, for example, the inertia weight is set to 0.9, the acceleration factor c1 value is 2.0, and the acceleration factor c2 value is 2.0. The particles in the first subpopulation are used to solve low-complexity problems.

[0064] S202b: Select particles from the target population to form a second sub-population in a manner that the inertia weight is randomly updated as the number of iterations increases and the acceleration factor is a constant.

[0065] In this embodiment, the random update of the inertia weight can be that the initial inertia weight is randomly generated in a preset value range, and the acceleration factor is also a fixed value, for example: the value range of the inertia weight is [0.4, 0.9]. The acceleration factor c1 value is 2.0, and the acceleration factor c2 value is 2.0. The particles in the second subpopulation are used to solve problems of medium complexity.

[0066] S202c: Select particles from the target population to form a third sub-population in a manner that the inertia weight is updated by the first formula and the acceleration factor is updated by the second formula as the number of iterations increases.

[0067] In this embodiment, the position update mode of the particles in the third sub-population refers to that the inertia weight and the acceleration factor change with the increase of the number of iterations. The particles in the third sub-population are suitable for solving high-complexity problems.

[0068] In an optional embodiment of the present application, the first formula is:

[0069] ω k =ω0·A(k)

[0070] In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population, A(k)=1-(1+e -k+N / 2 ) -1 , where e is a natural constant and N is the preset maximum number of iterations;

[0071] The second formula is:

[0072]

[0073] In the formula, c 1k is the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration.

[0074] In an optional embodiment of the present application, the first formula is:

[0075] ω k =ω0·A(k)

[0076] In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population, A(k)=1-(1+e -k+N / 2 ) -1 , where e is a natural constant and N is the preset maximum number of iterations.

[0077] The second formula is:

[0078]

[0079] In the formula, c 1k is the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration.

[0080] It can be seen from the above embodiments that the inertia weight and acceleration factor provided in the embodiments of the present application can be adaptively adjusted, and all formulas during adjustment include but are not limited to the above-mentioned first formula and second formula, and may also be other variations of the first formula and the second formula.

[0081] Based on the above embodiment, in an optional embodiment of the present application, the number and proportion of particles in the first sub-population, the second sub-population and the third sub-population are determined according to the complexity of the problem to be solved by the pre-stored neural network.

[0082] In this embodiment, the complexity of the problem to be solved by the pre-stored neural network is different, and the number and proportion of each type of particles in the corresponding target population may also be different. For example, when the complexity of the problem is high, the number of particles in the third sub-population in the target population and the proportion of all particles in the third sub-population to all particles in the target population should be increased accordingly. Conversely, when the complexity of the problem is reduced, the number of particles in the third sub-population in the target population and the proportion of all particles in the third sub-population to all particles in the target population can be reduced accordingly.

[0083] S203: Initialize the particles in each sub-population, update the initial vector of each particle, and determine the updated initial vector as the particle input vector, wherein the initial vector is determined according to the weights and thresholds contained in the pre-stored neural network.

[0084] In this embodiment, the initialization process refers to updating the initial vector corresponding to each particle for subsequent iterative solution. The initial vector refers to a one-dimensional vector consisting of the weights and thresholds of all particles obtained by integrating the weights and thresholds of each particle. For example: the pre-stored neural network contains 5 weights and 5 thresholds, then the corresponding initial vector is a one-dimensional vector integrated by 10 values.

[0085] Specifically, in an optional embodiment of the present application, in step S203, the particles in each sub-population are initialized, and updating the initial vector of each particle includes:

[0086] S203a: For each particle in each sub-population, a random number generation function is used to generate an initialization random number corresponding to the particle.

[0087] In this embodiment, the random number generation function may be a function for generating random numbers implemented by programming software, such as a rand function. For example, the random number generated by the random number generation function is 0.4.

[0088] S203b: Determine the numerical quantity of the initial vector according to the initial vector of the particle.

[0089] In this embodiment, the number of random numbers is determined according to the number of values ​​in the initial vector corresponding to the particle to be initialized. For example, if the initial vector is a vector containing 10 values, the number of values ​​in the initial vector is 10.

[0090] S203c: Update the initial vector of the particle according to the preset value range, number of values ​​and initialization random number.

[0091] In this embodiment, assuming that the pre-stored neural network contains 5 weights and 5 thresholds, the initial vector to be initialized is a one-dimensional vector containing 10 values, and each vector has a value range. If the weight value range in this embodiment is [0,1], and the threshold value range is [1,2], then according to the size of the 10 random numbers randomly generated before, the initial vector after initialization is [0.2, 0.3, 0.1, 0.9, 0.7, 1.2, 1.3, 1.5, 1.9, 1.3].

[0092] S204: Input the vector to be input to each particle, update the particle position of the particle according to the particle position update method corresponding to the particle, and obtain a new vector and fitness function value.

[0093] In this embodiment, the particle position update method of each subpopulation is different, so after inputting the input vector to each particle, the particle position is updated in accordance with the particle position update method, which is a process of calculating the function value of each particle using the fitness function and generating a new vector. The fitness function can be selected according to different neural networks.

[0094] In an optional embodiment of the present application, when the pre-stored neural network is a back propagation BP neural network, the fitness function value is the mean square error output by the BP neural network, and the corresponding fitness function is:

[0095]

[0096] Where f is the fitness function value, N is the number of BP neural network training samples, m is the number of output neurons, O ki is the expected output of the kth network for the ith sample, Y ki The k-th network predicts the output for the i-th sample.

[0097] When the pre-stored neural network is a learning vectorized LVQ neural network, the corresponding fitness function is:

[0098]

[0099] Where D(t) represents the average variance distance of the neural network output corresponding to the t-th particle, M represents the number of particles in the sub-population corresponding to the t-th particle, and X i (t) is the input vector corresponding to the t-th particle, P i (t) is a randomly generated vector, and f(t) is the fitness function value corresponding to the tth particle.

[0100] When the neural network BP neural network or LVQ neural network is pre-stored, the fitness function may also be improved by the corresponding fitness function formula in the above embodiment.

[0101] S205: According to the fitness function value corresponding to each particle in the target population, the historical optimal solution of each particle in each particle population and the sub-population optimal solution of the particle population are updated, and according to all the sub-population optimal solutions, the temporary global optimal solution of the target population is updated.

[0102] In this embodiment, the fitness function value corresponding to each particle is obtained by fitness function calculation, and then the fitness function values ​​of each particle in different sub-populations are compared, that is, the fitness function value currently obtained of the particle is compared with the fitness function value corresponding to the historical optimal solution obtained after the previous iteration to determine whether the new solution corresponding to the currently obtained fitness function value is better than the historical optimal solution obtained after the previous iteration. If so, the new solution corresponding to the currently obtained fitness function value is updated as the historical optimal solution to complete the update of the historical optimal solution of each particle. For example: the fitness function value of a particle in the first sub-population at the 4th iteration is a, and the fitness function value corresponding to the historical optimal solution of the previous 3 iterations is b, and the initial weight of the neural network needs to be optimized to a smaller value. Then, when a is less than b, it can be determined that the new solution obtained at the 4th iteration can be used as the new historical optimal solution of the previous 4 iterations, replacing the historical optimal solution corresponding to the fitness function value b after the previous 3 iterations.

[0103] In this embodiment, after all particles in each subpopulation update the historical optimal solution when the current iteration process is completed, the fitness function values ​​corresponding to the historical optimal solutions of each particle in the subpopulation are compared to obtain the historical optimal solution corresponding to the particle with the largest or smallest fitness function value in the subpopulation during the current iteration process and update it as the subpopulation optimal solution of the entire subpopulation. Finally, the same comparison method is used to compare the fitness function values ​​corresponding to the subpopulation optimal solutions of each subpopulation to update the global optimal solution of the entire target population.

[0104] S206: guiding the update of all particles through the temporary global optimal solution, taking the corresponding new vector as the input vector for the next iteration, returning to input the input vector to each particle, updating the particle position according to the particle position update method, and obtaining the new vector and fitness function value, until it is detected that the update iteration reaches the iteration stop condition, and outputting the temporary global optimal solution after the last iteration as the target global optimal solution, wherein the target global optimal solution is the target vector containing the weights and thresholds required by the pre-stored neural network.

[0105] In this embodiment, the temporary global optimal solution refers to the optimal solution in the optimal solution of each sub-population. The temporary global optimal solution can be used to guide the update of all particles in the entire target population, and then return to step S204 for the next iteration. The iteration stop condition can be that the fitness function value of a particle in the target population is less than a preset threshold and / or the number of iterations has reached a preset maximum number of iterations. At this time, the global optimal solution obtained by the last iteration at the end of the overall iterative process is output as the final solution. The global optimal solution is also a vector such as an initial vector, which contains relevant parameters of the neural network to be solved, such as the weights and thresholds required by the neural network.

[0106] S207: Determine the target weight and target threshold required by the pre-stored neural network according to the target vector.

[0107] In this embodiment, the process of determining the target weights and target thresholds required for the pre-stored neural network can be the inverse process of determining the input vector according to the weights and thresholds corresponding to each particle in step S203, so as to obtain the target weights and target thresholds required for the final pre-stored neural network.

[0108] Further, in an optional embodiment of the present application, before the particle position of the particle is updated according to the particle position update method corresponding to the particle in step S204 to obtain a new vector, the method further includes:

[0109] When a particle is detected to belong to the second sub-population, a random number generation function is used to generate a random number corresponding to the particle;

[0110] According to the random number corresponding to the particle and the preset inertia weight range, the inertia weight of the particle is randomly updated. The calculation formula is:

[0111] ω n =ω min +(ω max -ω min )·p

[0112] In the formula, ω n is the updated inertia weight, ω max is the maximum inertia weight in the preset inertia weight range, ω min is the minimum inertia weight in the preset inertia weight range, and p is the random number corresponding to the particle.

[0113] In this embodiment, as the iteration proceeds, for the second sub-population, the inertia weight is updated immediately at each iteration, so it is necessary to initialize the inertia weight before each iteration, randomly generate the inertia weight, and use the randomly generated inertia weight to guide the update of the particle position. In this embodiment, the random number generation function used is similar to the random number generation function in the above embodiment, so this embodiment will not be repeated here. For the particles in the first sub-population or the third sub-population, the inertia weight is not randomly generated, and there is no need to initialize the inertia weight.

[0114] Based on the above embodiment, in an optional embodiment of the present application, it also includes:

[0115] Step A: When a particle is detected to belong to the third sub-population, the inertia weight is divided into three change stages as the number of iterations increases; the three change stages are the first stage in which the inertia weight in the early stage of the iteration remains slightly less than the preset weight upper limit value, the second stage in which the inertia weight in the middle stage of the iteration gradually decreases as the iteration proceeds, and the third stage in which the inertia weight in the late stage of the iteration remains slightly greater than the preset weight lower limit value.

[0116] In this embodiment, in the early stage of iteration, the inertia weight is kept slightly less than the preset weight upper limit value, which means that the inertia weight should be kept within a larger numerical range, and the larger numerical range can be composed of some numerical values ​​approaching the preset weight upper limit value upward. A larger inertia weight is conducive to improving the global search ability of the particle swarm algorithm and ensuring the traversal of the particles to the entire solution space. As the iteration proceeds, the relationship weight of the particles should be gradually reduced to improve the search efficiency. In the later stage of iteration, the inertia weight of the particles is kept slightly greater than the preset weight lower limit value, which means that the inertia weight should be kept within a smaller numerical range, and the smaller numerical range can be composed of some numerical values ​​approaching the preset weight lower limit value downward. A smaller inertia weight can enhance the local search ability of the particle swarm algorithm.

[0117] In summary, the neural network parameter processing method provided in the embodiment of the present application first determines the parameters of the target population according to the pre-stored neural network, and then divides the target population into multiple sub-populations with different particle position update methods, thereby increasing the diversity of the target population, realizing information exchange in the process of generating the global optimal solution, and improving the search efficiency. In addition, an adaptive particle position update method is introduced in the iteration process to update the particle position, and by comparing the fitness function value of the particle, the historical optimal solution of each particle in each sub-population and the sub-population optimal solution of the sub-population are updated, and the temporary global optimal solution of the target population is updated, and finally the target global optimal solution is obtained at the end of the iteration, and the target weights and target thresholds required for the pre-stored neural network are determined according to the target global optimal solution, and the global optimization ability of the algorithm is improved in the process of solving the pre-stored neural network, thereby improving the overall accuracy of the algorithm, and being able to obtain a better initial weight of the neural network with less time cost, thereby saving time cost for the training process of the neural network.

[0118] At the same time, by adaptively adjusting the inertia weight and acceleration factor, the algorithm's global optimization capability is improved when solving problems with high complexity, and the accuracy of the algorithm is improved.

[0119] At the same time, by adjusting the number and proportion of particles in each sub-population in the target population, efficient calculation of problems of different complexity can be achieved, and the ability to process pre-stored neural network parameters with different requirements can be improved.

[0120] At the same time, by dividing the inertia weight into three stages of change as the number of iterations increases, and taking the corresponding numerical limit of the inertia weight in each stage of change, while ensuring the traversal of the particles in the entire solution space, it avoids the situation where the particles are already in the global optimal field but jump out of the field due to excessive particle speed, so as to achieve the purpose of improving search efficiency and search accuracy.

[0121] Figure 3 This is a structural diagram of a neural network parameter processing device provided in an embodiment of the present application, the device includes: a parameter definition module 31, a population division module 32, an initialization module 33 and a particle swarm iterative calculation module 34.

[0122] The parameter definition module 31 is used to determine the parameters of the target population according to the pre-stored neural network.

[0123] The population division module 32 is used to divide the target population into a plurality of sub-populations according to a preset position update method, wherein each sub-population includes at least two particles, and the particle position update methods of different sub-populations are different.

[0124] The initialization module 33 is used to divide the target population into multiple sub-populations according to a preset position update method, wherein each sub-population includes at least two particles, and the particle position update methods of different sub-populations are different.

[0125] The particle swarm iterative calculation module 34 is used to initialize the particles in each sub-swarm, update the initial vector of each particle, and determine the updated initial vector as the input vector of the particle, wherein the initial vector is determined according to the weights and thresholds contained in the pre-stored neural network.

[0126] The particle iterative calculation module 34 is further used to: input the vector to be input to each particle, update the particle position of the particle according to the particle position update method corresponding to the particle, and obtain a new vector and fitness function value.

[0127] The particle swarm iterative calculation module 34 is also used to update the historical optimal solution of each particle in each particle population and the sub-population optimal solution of the particle population according to the fitness function value corresponding to each particle in the target population, and update the temporary global optimal solution of the target population according to all the sub-population optimal solutions.

[0128] The particle swarm iterative calculation module 34 is also used to guide the update of all particles through a temporary global optimal solution, use the corresponding new vector as the input vector for the next iteration, return the input vector to each particle, update the particle position of the particle according to the particle position update method, and obtain the new vector and fitness function value until it is detected that the update iteration reaches the iteration stop condition, and output the temporary global optimal solution after the last iteration as the target global optimal solution, wherein the target global optimal solution is a target vector containing the weights and thresholds required by the pre-stored neural network.

[0129] The particle swarm iterative calculation module 34 is also used to determine the target weight and target threshold required by the pre-stored neural network according to the target vector.

[0130] In an optional embodiment of the present application, the preset position update method includes a change method of the inertia weight and the acceleration factor as the number of iterations increases; accordingly, the population division module 32 is specifically used to: select particles from the target population to form a first sub-population according to a change method in which the inertia weight and the acceleration factor are both constant values ​​as the number of iterations increases; select particles from the target population to form a second sub-population according to a change method in which the inertia weight is randomly updated and the acceleration factor is a constant value as the number of iterations increases; select particles from the target population to form a third sub-population according to a change method in which the inertia weight is updated with the first formula and the acceleration factor is updated with the second formula as the number of iterations increases.

[0131] In an optional embodiment of the present application, the first formula used by the population division module 32 is:

[0132] ω k =ω0·A(k)

[0133] In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population, A(k)=1-(1+e -k+N / 2 ) -1 , where e is a natural constant and N is the preset maximum number of iterations;

[0134] The second formula used is:

[0135]

[0136] In the formula, c 1k is the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration.

[0137] In an optional embodiment of the present application, the first formula used by the population division module 32 is:

[0138] ω k =ω0·(0.99) k

[0139] In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population;

[0140] The second formula is:

[0141]

[0142] In the formula, c 1k is the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration.

[0143] In an optional embodiment of the present application, the number and proportion of particles in the first sub-population, the second sub-population and the third sub-population divided by the population division module 32 are determined according to the complexity of the problem to be solved by the pre-stored neural network.

[0144] In an optional embodiment of the present application, the initialization module 33 is specifically used to: generate a random number corresponding to each particle in each sub-population using a random number generation function; determine the number of values ​​of the initial vector according to the initial vector of the particle; and update the initial vector of the particle according to a preset value range, number of values ​​and the initialization random number.

[0145] In an optional embodiment of the present application, the particle swarm iterative calculation module 34 is used to update the particle position of the particle according to the particle position update method corresponding to the particle, and before obtaining the new vector, it is also used to:

[0146] When a particle is detected to belong to the second sub-population, a random number generation function is used to generate a random number corresponding to the particle;

[0147] According to the random number corresponding to the particle and the preset inertia weight range, the inertia weight of the particle is randomly updated. The calculation formula is:

[0148] ω n =ω min +(ω max -ω min )·p

[0149] In the formula, ω n is the updated inertia weight, ω max is the maximum inertia weight in the preset inertia weight range, ω min is the minimum inertia weight in the preset inertia weight range, and p is the random number corresponding to the particle.

[0150] In an optional embodiment of the present application, in the particle swarm iterative calculation module 34, the inertia weight is divided into three change stages as the number of iterations increases; the three change stages are respectively a first stage in which the inertia weight in the early stage of the iteration remains slightly less than the preset weight upper limit value, a second stage in which the inertia weight in the middle stage of the iteration gradually decreases as the iteration proceeds, and a third stage in which the inertia weight in the late stage of the iteration remains slightly greater than the preset weight lower limit value.

[0151] The neural network parameter processing device provided in this embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0152] Figure 4 A schematic diagram of the hardware structure of a neural network parameter processing device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the device includes: at least one processor 401 and a memory 402 .

[0153] The memory 402 is used to store computer-executable instructions.

[0154] The processor 401 is used to execute the computer-executable instructions stored in the memory 402 to implement the various steps involved in the above method embodiment. For details, please refer to the relevant description in the above method embodiment.

[0155] Optionally, the memory 402 may be independent or integrated with the processor 401 .

[0156] When the memory 502 is independently provided, the device further includes a bus 403 for connecting the memory 502 and the processor 501 .

[0157] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above neural network parameter processing method is implemented.

[0158] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above neural network parameter processing method when executed by a processor.

[0159] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0160] The modules described above as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0161] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The unit composed of the above modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0162] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method of each embodiment of the present application.

[0163] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0164] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0165] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0166] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0167] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0168] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0169] The present invention is intended to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solution from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A neural network parameter processing method, characterized in that: Applicable to servers, including: Determining the parameters of the target population according to a pre-stored neural network; the pre-stored neural network is a neural network for inputting feature vectors obtained through a feature extraction process of a multi-beam seafloor sonar image; Dividing the target population into multiple sub-populations according to a preset position update method, wherein each sub-population includes at least two particles, and the particle position update methods of different sub-populations are different; Initializing the particles in each subpopulation, updating the initial vector of each particle, and determining the updated initial vector as the to-be-input vector of the particle, wherein the initial vector is determined according to the weights and thresholds included in the pre-stored neural network; Inputting the vector to be input into each particle, updating the particle position of the particle according to the particle position update method corresponding to the particle, and obtaining a new vector and fitness function value; According to the fitness function value corresponding to each particle in the target population, the historical optimal solution of each particle in each particle population and the sub-population optimal solution of the particle population are updated, and according to all the sub-population optimal solutions, the temporary global optimal solution of the target population is updated; Guide the update of all particles through the temporary global optimal solution, use the corresponding new vector as the vector to be input for the next iteration, return to the step of inputting the vector to be input to each particle, update the particle position of the particle according to the particle position update method corresponding to the particle, and obtain a new vector and a fitness function value, until it is detected that the update iteration reaches the iteration stop condition, and output the temporary global optimal solution after the last iteration as the target global optimal solution, wherein the target global optimal solution is a target vector containing the weights and thresholds required by the pre-stored neural network; According to the target vector, the target weight and target threshold required by the pre-stored neural network are determined.

2. The method according to claim 1, characterized in that The preset position update method includes the change method of the inertia weight and the acceleration factor as the number of iterations increases; Accordingly, the target population is divided into a plurality of sub-populations according to the preset position update method, wherein each sub-population includes at least two particles, including: Select particles from the target population to form the first sub-population in a manner that the inertia weight and the acceleration factor are constant as the number of iterations increases; Select particles from the target population to form a second sub-population in a manner that the inertia weight is randomly updated with the increase in the number of iterations and the acceleration factor is a constant value; Selecting particles from the target population to form a third subpopulation in a manner that the inertia weight is updated by the first formula and the acceleration factor is updated by the second formula as the number of iterations increases; The first formula is: oh k =ω0·A(k) In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population, A(k)=1-(1+e -k+N / 2 ) -1 , where e is a natural constant and N is the preset maximum number of iterations; The second formula is: In the formula, c 1k is the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration; Alternatively, the first formula is: oh k =ω0·(0.99) k In the formula, ω k is the inertia weight of the particles in the third sub-population at the kth iteration, ω0 is the initial weight of the particles in the third sub-population; The second formula is: In the formula, c 1k is the acceleration factor c1 value of the third type of particles at the kth iteration, c 2k is the acceleration factor c2 value of the third type of particles at the kth iteration, c 10 is the acceleration factor c1 value at the first iteration, c 20 is the acceleration factor c2 value at the first iteration.

3. The method according to claim 2, characterized in that The number and proportion of particles in the first sub-population, the second sub-population and the third sub-population are determined according to the complexity of the problem to be solved by the pre-stored neural network.

4. The method according to claim 2, characterized in that: The process of initializing the particles in each sub-population and updating the initial vector of each particle includes: For each particle in each sub-population, a random number generation function is used to generate an initialization random number corresponding to the particle; According to the initial vector of the particle, determining the numerical quantity of the initial vector; The initial vector of the particle is updated according to a preset value range, the number of values ​​and the initialization random number.

5. The method according to claim 2, characterized in that: Before updating the particle position of the particle according to the particle position update method corresponding to the particle to obtain a new vector, the method further includes: When a particle is detected to belong to the second sub-population, a random number generation function is used to generate a random number corresponding to the particle; According to the random number corresponding to the particle and the preset inertia weight range, the inertia weight of the particle is randomly updated, and the calculation formula is: oh n =ω min +(ω max -oh min )·p In the formula, ω n is the updated inertia weight, ω max is the maximum inertia weight in the preset inertia weight range, ω min is the minimum inertia weight in the preset inertia weight range, and p is the random number corresponding to the particle.

6. The method according to any one of claims 2 to 5, characterized in that Also includes: When a particle is detected to belong to the third sub-population, the inertia weight is divided into three change stages as the number of iterations increases; The three change stages are respectively the first stage in which the inertia weight in the early stage of iteration remains slightly less than the preset weight upper limit value, the second stage in which the inertia weight in the middle stage of iteration gradually decreases as the iteration proceeds, and the third stage in which the inertia weight in the late stage of iteration remains slightly greater than the preset weight lower limit value.

7. A neural network parameter processing device, characterized in that: include: A parameter definition module is used to determine the parameters of the target population according to a pre-stored neural network; the pre-stored neural network is a neural network for inputting a feature vector obtained by a feature extraction process of a multi-beam seafloor sonar image; A population division module is used to divide the target population into multiple sub-populations according to a preset position update method, wherein each sub-population includes at least two particles, and the particle position update methods of different sub-populations are different; An initialization module, used to initialize the particles in each sub-population, update the initial vector of each particle, and determine the updated initial vector as the input vector of the particle, wherein the initial vector is determined according to the weights and thresholds included in the pre-stored neural network; A particle swarm iterative calculation module is used to input the vector to be input to each particle, update the particle position of the particle according to the particle position update method corresponding to the particle, and obtain a new vector and fitness function value; The particle swarm iterative calculation module is used to update the historical optimal solution of each particle in each particle population and the sub-population optimal solution of the particle population according to the fitness function value corresponding to each particle in the target population, and update the temporary global optimal solution of the target population according to all the sub-population optimal solutions; The particle swarm iterative calculation module further guides the update of all particles through the temporary global optimal solution, uses the corresponding new vector as the vector to be input for the next iteration, returns the step of inputting the vector to be input to each particle, updates the particle position of the particle according to the particle position update method corresponding to the particle, and obtains a new vector and a fitness function value, until it is detected that the update iteration reaches the iteration stop condition, and outputs the temporary global optimal solution after the last iteration as the target global optimal solution, wherein the target global optimal solution is a target vector containing the weights and thresholds required by the pre-stored neural network; The particle swarm iterative calculation module is also used to determine the target weight and target threshold required by the pre-stored neural network according to the target vector.

8. A neural network parameter processing device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the neural network parameter processing method according to any one of claims 1 to 6.

Images