Inference based on neural network model, model training method and computing device cluster

Abstract

The embodiment of the application provides a neural network model-based reasoning method, a neural network model training method and a computing device cluster. The neural network model comprises N network layers and routing units corresponding to the network layers. Each network layer comprises a plurality of branch structures, and each branch structure has a corresponding processing function and processing parameters. The reasoning method comprises the following steps: determining initial input information, inputting the initial input information into the neural network model, determining at least one target branch structure in the corresponding network layer by each routing unit included in the neural network model according to target input information input into each routing unit in sequence, and performing reasoning processing on the target input information according to the processing function and the processing parameters corresponding to the target branch structure to obtain corresponding output information. In this way, when reasoning is performed, only part of the branch structures in the network layer can be selected to participate in the reasoning, the amount of calculation can be significantly reduced, resource consumption can be reduced, and reasoning efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a reasoning method based on a neural network model, a neural network model training method and a computing device cluster. BACKGROUND

[0002] With the wide application of deep learning in the fields of computer vision, natural language processing and the like, the scale and depth of neural network models are increasing. A neural network model usually includes multiple network layers, each network layer has a plurality of processing parameters corresponding to the network layer, and the neural network model obtains output information by using all the processing parameters of each network layer during reasoning.

[0003] Taking the application of a neural network model in the field of natural language processing as an example, input information input into the neural network model is sequentially processed by each network layer included in the neural network model, and each network layer performs corresponding reasoning processing based on all the processing parameters of the network layer to obtain an output result. That is, in this way, all input information will undergo completely fixed logical calculation, so that the input information input into the neural network model needs to use all the processing parameters of each network layer for reasoning in order to obtain an output result. Therefore, in the actual reasoning process, even if part of the processing parameters of the network layer participate in the reasoning processing, a relatively accurate reasoning result can be obtained, but since the neural network model is a fixed logical calculation process, the reasoning result will still be output after the reasoning processing by other processing parameters. In this way, there is a problem of redundant calculation, which not only leads to large resource consumption, but also leads to slow output of the corresponding output information, affecting the reasoning efficiency. Therefore, in the existing technology, in some reasoning scenarios, reasoning occupies a large amount of computing resources, and there is a problem of serious redundant calculation, causing resource waste and affecting reasoning efficiency. In addition, the neural network model also has the same problem when applied to other fields such as computer vision.

[0004] In summary, in the current existing technology, when reasoning based on the existing neural network model, there is a problem of redundant calculation, causing resource waste, affecting reasoning efficiency, and further affecting user experience. SUMMARY

[0005] The embodiments of the present application provide a reasoning method based on a neural network model, a neural network model training method and a computing device cluster. By branching the processing parameters in each network layer, the processing parameters included in any branch structure can be selected to perform reasoning processing on input information to obtain output information, without the need for the branch structure of the entire neural network model to participate in reasoning, that is, without the need for the parameters of the entire neural network model to participate in reasoning, thereby reducing computing resources, reducing redundant calculation, speeding up reasoning efficiency and improving user experience.

[0006] To solve the above technical problems, in a first aspect, embodiments of the present application provide a reasoning method based on a neural network model, the neural network model comprising N layers of network layers and a routing unit corresponding to each network layer, each network layer comprising at least one branch structure, each branch structure having a corresponding processing function and processing parameters, the method comprising: determining initial input information, inputting the initial input information to the neural network model, and determining, by the routing unit corresponding to each network layer of the neural network model, at least one target branch structure in the corresponding network layer according to target input information input to the routing unit, and inputting the target input information to the target branch structure, and performing reasoning processing on the target input information by the target branch structure according to the processing function and the processing parameters corresponding to the target branch structure, to obtain corresponding output information, wherein the target input information input to the routing unit corresponding to the first layer of network layers is the initial input information, the target input information input to the routing unit corresponding to the kth layer of network layers is the output information of the (k-1)th layer of network layers, and 1

[0007] With the above technical solution, by dividing the parameters of each layer of network layers of the neural network model, different branch structures are obtained, and by setting a corresponding routing unit for each layer of network layers, when the routing unit corresponding to each layer of network layers receives target input information, at least one branch structure is selected from the corresponding network layer as a target branch structure according to the target input information, and the target input information is processed by the target branch structure according to the corresponding processing function and processing parameters to obtain output information. In this way, only part of the parameters are involved in reasoning when reasoning, which can significantly reduce the amount of calculation, reduce resource consumption, improve reasoning efficiency, and improve user experience.

[0008] In a possible implementation of the above first aspect, the routing unit determines at least one target branch structure in the corresponding network layer according to the target input information input to the routing unit, comprising: determining matching information of the target input information and each branch structure included in the corresponding network layer, and determining the branch structure satisfying a target matching condition as the target branch structure according to the matching information.

[0009] With the above technical solution, the routing unit matches the branch structure suitable for processing the target input information according to the target input information to obtain the target branch structure, which ensures the accuracy of reasoning processing of each network layer on the target input information and ensures the accuracy of the final output information.

[0010] In a possible implementation of the first aspect, the routing unit comprises a lightweight neural network, and the routing unit determines matching information of the target input information and each branch structure included in the corresponding network layer, and determines, as the target branch structure, a branch structure satisfying a target matching condition according to the matching information, by: determining, based on the lightweight neural network, an activation score or a probability distribution of each branch structure corresponding to the target input information as the matching information; and determining, based on a TOP-K selection manner, the target branch structure according to the matching information.

[0011] By adopting the above technical solution, the activation score or the probability distribution of each branch structure corresponding to the target input information is determined based on the lightweight neural network, and a more suitable branch structure is selected according to the activation score or the probability distribution based on the TOP-K selection manner, which can improve the branch selection efficiency and flexibility, and ensure the accuracy of the inference processing based on the at least one target branch structure that is most matched.

[0012] In a possible implementation of the first aspect, the routing unit determines matching information of the target input information and each branch structure included in the corresponding network layer, and determines, as the target branch structure, a branch structure satisfying a target matching condition according to the matching information, by: determining, based on a similarity measurement function, a similarity of each branch structure corresponding to the target input information as the matching information; and determining, based on a TOP-K selection manner, the target branch structure according to the matching information.

[0013] By adopting the above technical solution, the similarity of each branch structure corresponding to the target input information is determined based on the similarity measurement function, and the target branch structure is selected according to the similarity based on the TOP-K selection manner, so that the routing unit can be directly implemented without additional training, and the best branch structure selection in a computing resource limited scenario can be realized based on the similarity calculation, which is an efficient and highly interpretable selection strategy and can ensure the accuracy of the inference processing of the target input information.

[0014] In a possible implementation of the first aspect, the routing unit determines matching information of the target input information and each branch structure included in the corresponding network layer, and determines, as the target branch structure, a branch structure satisfying a target matching condition according to the matching information, by: determining, based on a hash function, a mapping result of each branch structure corresponding to the target input information as the matching information; and determining, as the target branch structure, the branch structure satisfying the target matching condition according to the matching information.

[0015] According to the technical scheme, the mapping result of the target input information corresponding to each branch structure is determined based on the hash function, and the target branch structure is determined according to the mapping result, which is significantly advantageous in branch structure selection efficiency and scalability, and is a branch selection method with high efficiency and low calculation cost in the case of ensuring that the most matched branch structure can be found.

[0016] In a possible implementation of the first aspect, the determining, by the routing unit, of the at least one target branch structure in the corresponding network layer according to the target input information input into the routing unit comprises: determining a similarity between the target input information and historical input information; if it is determined that there is historical input information that meets a preset similarity condition, taking the historical input information as target historical input information, and taking a branch structure that performs inference processing on the target historical input information as the target branch structure; and if it is determined that there is no historical input information that meets the preset similarity condition, determining matching information of the target input information and each branch structure included in the corresponding network layer, and taking a branch structure that meets a target matching condition as the target branch structure according to the matching information.

[0017] According to the technical scheme, if the neural network model has processed similar input information, inference processing of current input information is directly performed based on a branch path of the similar input information, without the need to perform branch structure selection calculation again, thereby reducing calculation resources and further accelerating inference efficiency.

[0018] In a possible implementation of the first aspect, after the target branch structure is determined, the method further comprises: activating the target branch structure, so that the target branch structure performs inference processing on the target input information according to a processing function and a processing parameter corresponding to the target branch structure; and wherein the branch structures are arranged in parallel, and the processing functions corresponding to the branch structures are the same and the processing parameters are different, or both the processing functions and the processing parameters are different.

[0019] According to the above scheme, the branch structure is activated only when the branch structure needs to participate in inference, thereby effectively reducing the operation resources of the neural network model, and by dividing the parameters, different branch structures can implement the same processing function or different processing functions based on different parameters.

[0020] In a second aspect, the implementation of the present application also discloses a neural network model training method, which comprises the following steps: determining an initial neural network model, the initial neural network model comprising N network layers and routing units corresponding to the network layers, each network layer comprising at least one branch structure, each branch structure having a corresponding processing function and processing parameters; determining a training sample set and a training verification set; based on a soft selection mechanism, performing forward propagation training and backward propagation training on each branch structure included in each network layer of the initial neural network model and the routing units corresponding to the network layers according to the training sample set, to obtain a first neural network model; determining a training result of the first neural network model according to the training verification set; when the training result meets a preset training condition, performing inference optimization on the first neural network model according to the training sample set based on a hard selection mechanism, to obtain a target neural network model, the target neural network model being used to implement the inference method based on the neural network model provided in any one of the implementation modes of the first aspect.

[0021] By adopting the above technical solution, the neural network model is first trained based on the soft selection mechanism to ensure that each branch structure can be trained, and then the hard selection mechanism is switched to optimize the neural network model, so that the performance of the trained neural network model is more optimal.

[0022] In a possible implementation of the second aspect, the processing functions of the branch structures are the same, the processing parameters are different, or both the processing functions and the processing parameters are different.

[0023] In a possible implementation of the second aspect, based on the soft selection mechanism, the forward propagation training and the backward propagation training are performed on each branch structure included in each network layer of the initial neural network model according to the training sample set, which comprises the following steps: determining the similarity between each training sample included in the training sample set; based on the soft selection mechanism, a path consistency regularization method is adopted to select training samples satisfying a target similarity condition to perform the forward propagation training and the backward propagation training on the same branch structure.

[0024] By adopting the above technical solution, the path consistency regularization method is adopted to select training samples satisfying a target similarity condition to perform the forward propagation training and the backward propagation training on the same branch structure, so as to ensure the consistency of the calculation path of similar training samples and improve the generalization ability of the neural network model.

[0025] In a possible implementation of the second aspect, the first neural network model is optimized based on the hard selection mechanism to obtain a target neural network model, including: based on the hard selection mechanism, each routing unit adopts a TOP-K selection manner or an Argmax selection manner, a training sample set is selected to select at least one branch structure included in a corresponding network layer as a target branch structure, and the target branch structure is activated to enable the target branch structure to perform inference processing on the training sample set according to a corresponding processing function and processing parameter to obtain an output result, and the first neural network model is optimized according to the output result to obtain the target neural network model.

[0026] By using the technical solution, the ability of the routing unit to select a branch structure and the ability of the branch structure to process target input information are optimized based on the hard selection mechanism, and the performance of the neural network model is improved.

[0027] In a third aspect, the implementation of the present application further discloses a computing device cluster, including: at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the inference method based on the neural network model provided by any one of the implementation manners of the first aspect, or so that the computing device cluster executes the neural network model training method provided by any one of the implementation manners of the second aspect.

[0028] By using the technical solution, the neural network model training and / or inference processing based on the neural network model are performed based on multiple computing devices, parallel computing is achieved, and the training efficiency and the inference efficiency are improved.

[0029] In an implementation manner of the third aspect, at least one network layer of N network layers included in the neural network model and at least one branch structure included in the at least one network layer are deployed on each computing device.

[0030] By using the technical solution, each network layer and the branch structure of each network layer are deployed on different computing devices based on a distributed deployment manner, parallel computing is achieved, and the inference efficiency is further improved.

[0031] In a fourth aspect, the implementation of the present application further discloses a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program can be executed by a computer cluster to implement the inference method based on the neural network model provided by any one of the implementation manners of the first aspect, and / or the neural network model training method provided by any one of the implementation manners of the second aspect.

[0032] In a fifth aspect, the implementation of the present application further discloses a computer program product comprising a computer program, which, when executed by a computer cluster, implements the inference method based on the neural network model provided in any one of the implementation manners of the first aspect, and / or the neural network model training method provided in any one of the implementation manners of the second aspect.

[0033] The related beneficial effects of the third aspect to the fifth aspect can be referred to the related description in the first aspect or the second aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the present application, the drawings used in the implementation description will be briefly introduced as follows.

[0035] Figure 1 A flowchart of a neural network model training method provided by an embodiment of the present application;

[0036] Figure 2 A structure diagram of a network layer included in a neural network model provided by an embodiment of the present application;

[0037] Figure 3 A flowchart of an inference method based on a neural network model provided by an embodiment of the present application;

[0038] Figure 4 Another flowchart of an inference method based on a neural network model provided by an embodiment of the present application;

[0039] Figure 5 A flowchart of a target branch structure determination method provided by an embodiment of the present application;

[0040] Figure 6 Another flowchart of a target branch structure determination method provided by an embodiment of the present application;

[0041] Figure 7 Another flowchart of a target branch structure determination method provided by an embodiment of the present application;

[0042] Figure 8 A structure diagram of a computing device provided by an embodiment of the present application;

[0043] Figure 9 A structure diagram of a computing device cluster provided by an embodiment of the present application. DETAILED DESCRIPTION

[0044] With the wide application of deep learning in computer vision, natural language processing and recommendation system, the scale and depth of neural network models are increasing. However, the expansion of network scale and the increase of network depth make the neural network model include more and more network layers, and the corresponding processing parameters of each network layer are more and more, and the inference process based on the neural network model is also more and more complex.

[0045] Taking the application of neural network model in natural language processing field for user dialogue processing as an example, the existing inference processing mode is to input the user dialogue information as input information into the neural network model, and sequentially pass through each layer network layer included in the neural network model. The forward inference calculation and the backward inference calculation are performed by each layer network layer according to the corresponding all processing parameters. After the corresponding inference calculation is performed by the last layer network layer according to the corresponding all processing parameters, the output result is obtained and output to the user.

[0046] However, in the actual inference process, many input information may only need to be inferred based on part of the processing parameters to obtain accurate output results, and do not need all processing parameters to participate in the calculation. However, since the inference process of the neural network model is a completely fixed logic calculation, each layer network layer uses all parameters for inference calculation, which will cause a large amount of calculation resources to be occupied in the inference and neural network model training process. Moreover, in the actual inference process, many input information only need to activate part of the processing parameters in the network to obtain accurate output results. The participation of all parameters in the calculation will not only cause calculation redundancy, but also affect the accuracy of the output results. Therefore, due to the complexity of the existing neural network model calculation, in edge devices or low-power devices, for example, due to the disadvantages of the existing inference mode, it is difficult to balance the calculation speed and energy efficiency.

[0047] In order to solve the above problems, the existing technology also proposes to reduce the scale of processing parameters of the neural network model by network pruning, sparse training and other technologies. However, this way, when the neural network model processes complex problems, due to the reduction of the scale of processing parameters, there will be a problem that accurate output results cannot be obtained. And network pruning, sparse training and other technologies are all post-processing optimization or static sparsification of the neural network model, and cannot realize dynamic path selection depending on the input. There are still deficiencies in the flexibility and efficiency of the neural network model in processing different input information.

[0048] Based on this, the application provides a neural network model training method and an inference method based on the neural network model. In the initial stage of training the neural network model, the processing parameters corresponding to each network layer included in the initial neural network model are divided, so that each network layer includes at least one branch structure. Each branch structure is trained based on a training sample set and a training verification set, so that each branch structure of each network layer implements corresponding inference processing based on corresponding processing functions and processing parameters. In this way, during the inference process based on the neural network model, at least one branch structure in each network layer can be selected for inference processing according to input information, without the need for all processing parameters to participate in operation. In the case where the performance of the neural network model is not reduced, the inference efficiency is effectively improved, and the user experience can be greatly improved.

[0049] Next, the neural network model training method provided by the implementation manner of the application will be described in detail.

[0050] As shown in Figure 1 , the neural network model training method provided by the implementation manner of the application includes the following steps.

[0051] S10, determine an initial neural network model, the initial neural network model includes N network layers and routing units corresponding to each network layer, each network layer includes at least one branch structure, and each branch structure has corresponding processing functions and processing parameters.

[0052] For example, the processing parameters corresponding to each network layer included in the initial neural network model are changed from traditional full connection or convolutional layers to multi-branch structures, each branch structure has corresponding processing functions and a group of independent processing parameters. Each branch structure contains corresponding operation units to implement corresponding processing functions.

[0053] As shown in Figure 2 , a multi-layer network layer schematic diagram of a neural network model provided by the application is shown, each layer of network layer includes one or more branch structures, as shown in the figure, some branch structures (routing units are not shown) of the i-th layer to the i+4-th layer of network layer are shown, wherein each dot represents a branch structure. For example, the dot in the i-th layer of network layer is a branch structure, the four dots in the i+1-th layer of network layer are four parallel branch structures, the four dots in the i+2-th layer of network layer are four parallel branch structures, the four dots in the i+3-th layer of network layer are four parallel branch structures, and the dot in the i+4-th layer of network layer is a branch structure.

[0054] Further, the lines connecting each dot indicate the information transmission between the corresponding branch structures, representing the optional path between the branch structures. The path represented by the black thick solid line represents the selected calculation path, and the dot in the path is the target branch structure.

[0055] Thus, based on the target input information, the target branch structure is obtained by matching the i th network layer to the i+4 th network layer, and a calculation path is formed.

[0056] That is, in the implementation of the present application, some network layers in the N network layers can include a branch structure.

[0057] In the implementation of the present application, the network layer including a branch structure can have a corresponding routing unit, or can not have a corresponding routing unit.

[0058] In an implementation of the present application, each network layer includes a plurality of parallel branch structures, and each network layer has a corresponding routing unit. Each branch structure includes an operation unit and a set of independent processing parameters to implement a corresponding processing function.

[0059] S20, determining a training sample set and a training verification set, based on a soft selection mechanism, performing forward propagation training and backward propagation training on each branch structure included in each network layer included in the initial neural network model and the routing unit corresponding to each network layer according to the training sample set, to obtain a first neural network model.

[0060] S30, determining the training result of the first neural network model according to the training verification set, and when the training result meets a preset training condition, performing inference optimization on the first neural network model based on a hard selection mechanism according to the training sample set to obtain a target neural network model.

[0061] The neural network model training method provided by the implementation of the present application divides the processing parameters corresponding to each network layer included in the initial neural network model to generate at least one branch structure, each branch structure has a corresponding processing function and processing parameters, and each network layer is provided with a corresponding routing unit. Based on a soft selection mechanism, forward propagation training and backward propagation training are performed on each routing unit and each branch structure included in the corresponding network layer according to the training sample set, so that the training gradient can be propagated to multiple potential branch structures to obtain a first neural network model. So that each routing unit of the first neural network model has the ability to select a branch structure according to input information, and each branch structure has the ability to independently process input information based on the corresponding processing function and processing parameters. Further, after determining that the training result of the first neural network model meets the preset training condition according to the training verification, the hard selection mechanism is switched to, and the first neural network model is optimized based on the hard selection optimization to obtain a neural network model with better inference effect.

[0062] In the implementation of the present application, each branch structure has the same processing function and different processing parameters, or both the processing function and the processing parameters are different.

[0063] Exemplarily, the branch structures can have same or different functions, and the processing parameters corresponding to the branch structures can be partially same or completely different, i.e., the processing parameters of the branch structures can cross.

[0064] For example, the processing functions of the branch structures can be same convolution operation functions or full connection operation functions, or can be processing functions corresponding to different kernel sizes or processing functions corresponding to different feature dimensions.

[0065] In the implementation of the present application, based on the soft selection mechanism, the forward propagation training and the backward propagation training are performed on each branch structure included in each network layer of the initial neural network model according to the training sample set, including: determining the similarity between each training sample included in the training sample set; based on the soft selection mechanism, the path consistency regularization method is used to select the training samples satisfying the target similarity condition to perform the forward propagation training and the backward propagation training on the same branch structure.

[0066] Exemplarily, a small neural network is preselected to be constructed, and the branch feature information of each branch structure of the current layer is used as the input, and the initial scores of the branch structures are output. The Softmax function is used to convert the initial scores of the branch structures into a probability distribution, wherein the sum of the probabilities of all branch structures is 1. A temperature coefficient is introduced, wherein the temperature coefficient is initially set to 1.0 to realize a smooth distribution, and is gradually reduced to 0.1 in the training process, so that the probability distribution approaches a discrete decision state (for example, in One-hot form), and prepares for switching to a hard selection mechanism.

[0067] Specifically, a routing warm-up phase is first performed, including fixing the processing parameters corresponding to the branch structures of the main network layer, and only training the routing unit, so that the routing unit preliminarily learns the training sample distribution mode. Then, a joint optimization phase is performed, all processing parameters are unfrozen, and the routing unit and each branch structure are fine-tuned together.

[0068] Further, the value of the temperature coefficient is reduced in a linear or exponential manner, for example, the temperature coefficient is reduced by 0.1 every 10 epochs, and the entropy value of the routing unit is monitored, and the temperature coefficient is stopped to decay when the entropy value is lower than a preset entropy threshold (for example, 0.3).

[0069] Further, a higher learning rate (for example, a learning rate higher than a preset multiple of the learning rate of the corresponding network layer) is used for the routing unit, so as to prevent the processing parameters of the branch structures from dominating the optimization direction.

[0070] Further, a path consistency regularization manner is adopted, similarity of each training sample is calculated, and multiple similar training samples with similarity greater than a preset similarity threshold (for example, 80%, 85%, 90%, or 95%) (as an example of similarity meeting a target similarity condition) are forced to be input to the same branch structure corresponding to the same training path, so that similar training samples can select consistent paths in each network layer as much as possible, the generalization ability of the neural network model is improved, and the adversarial robustness of the neural network model is enhanced.

[0071] In the implementation of the present application, the similarity of each training sample can be cosine similarity, Euclidean similarity, or Mahalanobis similarity.

[0072] Further, in the training process, the activation frequency of each branch structure is updated, and the branch structure that has not been activated for a long time is removed, so as to avoid that the long-idle branch occupies the resources of the neural network model.

[0073] In this way, the neural network model training based on the soft selection mechanism can fully explore the combination potential of each branch structure in the training stage, so that each branch structure and the combination of each branch structure can have inference ability (i.e., the neural network model is differentiable or derivable) for forward inference and backward inference of input information, thereby laying a foundation for efficient inference.

[0074] Further, after the inference training of each branch structure based on the soft selection mechanism and the path consistency regularization, the training accuracy of the first neural network model obtained by training and the entropy value of each routing unit (as an example of training result) are determined according to a training verification set. If the training accuracy is greater than a preset accuracy threshold (for example, greater than 80%) and the entropy value of each routing unit is less than a preset entropy threshold (for example, less than 0.3) (as an example of training result meeting a preset training condition), it is indicated that the first neural network model has converged and the decision of each routing unit is clear enough. The hard selection mechanism can be switched to for inference optimization of the first neural network model.

[0075] In the implementation of the present application, the first neural network model is optimized based on the hard selection mechanism to obtain a target neural network model, including: based on the hard selection mechanism, each routing unit adopts a TOP-K selection manner or an Argmax selection manner, at least one branch structure included in a corresponding network layer is selected as a target branch structure according to a training sample set, and the target branch structure is activated, so that the target branch structure performs inference processing on the training sample set according to the corresponding processing function and processing parameter to obtain an output result. The branch structure selection ability of each routing unit is optimized, and the inference ability of each branch structure is optimized to obtain the target neural network model.

[0076] Further, in the implementation of the present application, a sparse subgraph is generated according to the selected target branch structure, so that the user can monitor the training process according to the sparse subgraph and modify the processing parameters of the neural network model according to the sparse subgraph.

[0077] The neural network model training method provided by the implementation of the present application, that is, a neuron-level super-sparse network design method, can obtain a multi-layer branch sparse neural network structure, can maintain gradient propagation to all potential branches based on a soft selection mechanism, allows the neural network model to explore combinations of different branch structures, avoids the gradient breaking problem caused by discrete decision-making in early training, gradually switches to a hard selection mechanism in a progressive manner, so that the neural network model gradually adapts to sparsification and reduces the performance mutation risk caused by direct hard selection. In addition, the path consistency regularization method is used to force similar input information to activate consistent paths (i.e., activate the same branch structure) in multiple network layers, reduce routing jitter, make the neural network model more robust to input information disturbance, enhance adversarial robustness, and enable each branch structure to focus on processing similar problems, improve the utilization rate of each branch structure of the neural network model, and make each branch structure processing input information professional.

[0078] Further, referring to Figure 3 The implementation of the present application also provides a reasoning method based on a neural network model, which can be applied to the fields of computer vision, natural language processing, and user recommendation systems.

[0079] S100, determining initial input information, and inputting the initial input information into the neural network model.

[0080] S200, sequentially determining at least one target branch structure in each network layer corresponding to the routing unit of the neural network model according to the target input information input into the routing unit.

[0081] S300, inputting the target input information into the target branch structure, and performing reasoning processing on the target input information according to the processing function and the processing parameter corresponding to the target branch structure, to obtain corresponding output information,

[0082] Wherein, the target input information input into the routing unit of the first layer network layer is the initial input information, and the target input information input into the routing unit of the kth layer network layer is the output information of the (k-1)th layer network layer, 1

[0083] The inference method based on a neural network model provided in this application divides the parameters of each network layer of the neural network model to obtain different branch structures. By setting a corresponding routing unit for each network layer, when the routing unit receives target input information, it selects at least one branch structure from the corresponding network layer as the target branch structure. This target branch structure then performs inference processing on the target input information according to its corresponding processing function and parameters to obtain output information. Thus, during inference, only a portion of the parameters are required, significantly reducing computational load, lowering resource consumption, improving inference efficiency, and enhancing the user experience.

[0084] Taking natural language application scenarios as an example, the initial input information is user input information, such as at least one of the following: text information entered by the user through a human-computer interaction interface, voice information entered by the user through voice input, and file information entered by the user through file transfer. The output information is the response information based on the user input information.

[0085] Furthermore, such as Figure 4 As shown, the inference method based on a neural network model provided in this application includes the following steps.

[0086] S1, input the initial input information into the routing unit corresponding to the first layer of the neural network model.

[0087] S2, the routing unit corresponding to the first network layer determines at least one branch structure in the first network layer as the target branch structure based on the initial input information, inputs the initial input information into the target branch structure, and the target branch structure performs reasoning processing on the initial input information according to the processing function and processing parameters corresponding to the target branch structure to obtain the corresponding output information, and inputs the output information as the target input information into the routing unit corresponding to the next network layer.

[0088] S3, the routing unit corresponding to the k-th network layer determines at least one target branch structure in the k-th network layer based on the input target information, inputs the initial input information into the target branch structure, and the target branch structure performs reasoning processing on the initial input information according to the processing function and processing parameters corresponding to the target branch structure to obtain the corresponding output information. The output information is then used as the target input information and input into the routing unit corresponding to the next network layer, where 1 < k ≤ N-1 and N is greater than or equal to 2.

[0089] S4, until the routing unit corresponding to the Nth network layer determines at least one target branch structure in the Nth network layer according to the input target input information, and inputs the initial input information into the target branch structure, and the target branch structure performs inference processing on the initial input information according to the processing function and the processing parameter corresponding to the target branch structure, to obtain corresponding output information.

[0090] Further, in the implementation of the present application, after the initial input information is input into the neural network model, feature extraction is further performed on the initial input information to obtain initial input feature information. That is, the target input information input into each network layer is target input feature information.

[0091] In an implementation of the present application, the routing unit determines at least one target branch structure in the corresponding network layer according to the input target input information, including: determining the similarity between the target input information (i.e., target input feature information) and historical input information (i.e., historical input feature information), and if it is determined that there is historical input information that meets the preset similarity condition, taking the historical input information as target historical input information (i.e., target historical input feature information), and taking the branch structure that performs inference processing on the target historical input information as the target branch structure.

[0092] For example, in an implementation, the routing unit corresponding to the first network layer calculates the similarity between the initial input information and the historical input information, and if it is determined that there is historical input information (as an example of meeting the preset similarity condition) that meets the similarity greater than a preset similarity threshold (for example, 90%, 92%, 95%, 98%, 99%), the historical input information is taken as target historical input information, and the branch structure corresponding to each network layer that performs inference processing on the target historical input information is determined as the target branch structure corresponding to each network layer. In this way, for similar input information, the branch structure that processes similar historical input information can be directly determined by the routing unit of the first layer, a sparse activation path is formed in the entire network based on the branch structure, the determined processing path is activated to activate the corresponding branch structure in each network layer, inference is performed on the target input feature information to obtain corresponding output information, and finally the final output result is output by the last network layer.

[0093] In another implementation, the routing unit corresponding to each network layer determines the similarity between the target input information and the historical input information, and if it is determined that there is historical input information that meets the similarity greater than a preset similarity threshold (for example, 90%, 92%, 95%, 98%, 99%), the historical input information is taken as target historical input information corresponding to each network layer, and the branch structure that performs inference processing on the target historical input information in the network layer is determined as the target branch structure corresponding to the network layer.

[0094] For example, the routing unit corresponding to the first layer network layer calculates the similarity between the initial input information and the historical input information, and if it is determined that there is historical input information satisfying the similarity condition (e.g., the similarity is greater than a preset similarity threshold value (e.g., 90%, 92%, 95%, 98%, 99%)), the historical input information is determined as the target historical input information corresponding to the first layer, the branch structure corresponding to the target historical input information in the first layer network layer is determined as the target branch structure corresponding to the first layer network layer, the initial input feature information is processed by the operation unit of the target branch structure of the first layer network layer according to the processing function and the processing parameter corresponding to the target branch structure to obtain the output information, and the output information of the first layer network layer is taken as the target input information of the second layer network layer. The routing unit of the second layer calculates the similarity between the target input information and the historical input information corresponding to the second layer, and if it is determined that there is historical input information satisfying the similarity condition, the historical input information is determined as the target historical input information corresponding to the second layer, the branch structure corresponding to the target historical input information in the second layer network layer is determined as the target branch structure corresponding to the second layer network layer, the initial input feature information is processed by the operation unit of the target branch structure of the second layer network layer according to the processing function and the processing parameter corresponding to the target branch structure to obtain the output information. In this way, although the input information is similar, the output information obtained by processing the target input feature information in the target branch structure of each layer network layer may be different from the output information corresponding to the target historical input information. Therefore, the routing unit of each layer determines the branch structure of the historical input information satisfying the similarity condition according to the target input feature information (i.e., the output information of the previous layer network layer) corresponding to the layer. In this way, even if the input information is similar, the branch path of the target branch structure of the entire network layer determined for the similar input information may be different from the historical branch path of the similar historical input information, which can make the inference processing of the input information more flexible and more accurate.

[0095] Further, if it is determined that there is no historical input information satisfying the preset similarity condition, the matching information between the target input information and each branch structure included in the corresponding network layer is determined, and the branch structure satisfying the target matching condition is determined as the target branch structure according to the matching information.

[0096] For example, if the routing unit corresponding to the first layer network layer determines that there is no historical input information satisfying the similarity greater than the preset similarity threshold, the matching information of the initial input information and each branch structure in the first layer network layer is determined according to the initial input information, the branch structure satisfying the target matching condition is taken as the target branch structure corresponding to the first layer network layer according to the matching information, the initial input feature information is processed by the operation unit of the target branch structure of the first layer network layer according to the processing function and the processing parameter corresponding to the target branch structure, the output information is obtained, and the output information of the first layer network layer is taken as the target input information of the second layer network layer. The routing unit of the second layer determines whether there is historical input information satisfying the preset similarity condition according to the target input information corresponding to the layer, and determines the target branch structure in the layer network layer according to the determination result. In this way, the target branch structure in the last layer network layer is selected by the routing unit of the last layer according to the target input information corresponding to the layer, the operation unit of the target branch structure of the last layer network layer processes the target input information according to the processing function and the processing parameter corresponding to the target branch structure, and finally obtains the output information. The output information is output to the user.

[0097] That is, in the implementation manner of the application, for an initial input information, each layer routing unit performs similarity calculation and similar historical input information matching according to the target input information corresponding to the layer, and a branch path including a branch structure processing target historical input information and a target branch structure determined according to the matching degree is formed.

[0098] In another implementation manner of the application, the similarity matching of the historical input information can also not be performed. When the input information is input into the neural network model, the routing unit corresponding to each layer network layer directly determines the matching information of the target input information and each branch structure included in the corresponding network layer according to the target input information input into the routing unit, and takes the branch structure satisfying the target matching condition as the target branch structure according to the matching information. That is, in another implementation manner of the application, the routing unit determines at least one target branch structure in the corresponding network layer according to the target input information input into the routing unit, including: determining the matching information of the target input information and each branch structure included in the corresponding network layer, and taking the branch structure satisfying the target matching condition as the target branch structure according to the matching information.

[0099] Exemplarily, for each network layer, the matching information of the target input information and each branch structure in the network layer is determined by the routing unit corresponding to the network layer according to the target input information (i.e., target input feature information), at least one branch structure satisfying the matching condition is taken as the target branch structure of the network layer according to the matching information, the output information is obtained by the operation unit included in the target branch structure according to the processing function and the processing parameter corresponding to the target branch structure, and the finally obtained output information is output to the user by the last network layer.

[0100] In an implementation manner, the routing unit corresponding to each network layer includes a lightweight neural network, such as Figure 4 As shown, the routing unit corresponding to each network layer determines the matching information of the target input information and each branch structure included in the corresponding network layer, and takes the branch structure satisfying the target matching condition as the target branch structure according to the matching information, including the following steps.

[0101] S210, based on the lightweight neural network, the activation score or probability distribution of the target input information corresponding to each branch structure is determined as the matching information.

[0102] Exemplarily, the lightweight neural network adopts 1-2 fully connected layers + activation function (for example, ReLU) to take the target input feature information and the feature map of the branch structure as input and output the activation score (Logits) or probability distribution of each branch structure.

[0103] S220, based on the TOP-K selection manner, the branch structure satisfying the target matching condition is taken as the target branch structure according to the matching information.

[0104] Exemplarily, the K branch structures with the highest activation score or the largest probability distribution are taken as the target branch structure based on the TOP-K selection manner, , and M is the number of branch structures in the corresponding network layer.

[0105] In another implementation manner, as shown in Figure 5 The routing unit corresponding to each network layer determines the matching information of the target input information and each branch structure included in the corresponding network layer according to the matching information, and takes the branch structure satisfying the preset matching condition as the target branch structure, including the following steps.

[0106] S230, based on the similarity measure function, the similarity of the target input information corresponding to each branch structure is determined as the matching information.

[0107] Exemplarily, each branch structure is associated with a learnable branch feature information, and for the input feature information corresponding to the target input information, the similarity of the branch feature information of each branch structure of the input feature information is calculated.

[0108] wherein the similarity is calculated by:

[0109]

[0110] wherein, is the input feature information and branch feature information of branch structure i the similarity of is a cosine similarity, Euclidean similarity, Mahalanobis similarity calculation function.

[0111] S240, based on the TOP-K selection manner, the branch structure meeting the target matching condition is selected as the target branch structure according to the matching information.

[0112] For example, based on the TOP-K selection manner, the K branch structures with the highest similarity are selected as the target branch structure.

[0113] In another implementation manner, as shown in Figure 6 , the routing unit corresponding to each layer of the network layer determines the matching information of the target input information and each branch structure included in the corresponding network layer, and selects the branch structure meeting the target matching condition as the target branch structure according to the matching information, including the following steps.

[0114] S250, based on the hash function, the mapping result of the target input information corresponding to each branch structure is determined as the matching information.

[0115] For example, the target input feature information is mapped to discrete hash buckets (M is the number of branch structures of the corresponding network layer, and the number of branch structures is the same or different in different network layers), that is .

[0116] Depending on the design of the hash function, similar target input feature information will be mapped to the same or adjacent buckets.

[0117] wherein the hash function is one or a combination of local sensitive hash, consistent hash and Learned Hashing.

[0118] S260, according to the matching information, the branch structure meeting the target matching condition is selected as the target branch structure.

[0119] For example, if it is a single hash function, the branch structure corresponding to the mapped hash bucket is selected as the target branch structure. If it is a multi-hash function, the branch structure corresponding to each hash bucket mapped by the combination of multiple hash functions is selected as the target branch structure.

[0120] Further, in the implementation of the present application, for each layer of the network layer target branch structure, after the target branch structure is determined, the method further comprises: activating the target branch structure to enable the target branch structure to perform inference processing on the target input information according to the processing function and the processing parameter corresponding to the target branch structure.

[0121] In the implementation of the present application, the processing function of each branch structure is the same, the processing parameter is different, or both the processing function and the processing parameter are different.

[0122] For example, when each layer of the neural network model does not perform inference processing, the processing parameter corresponding to each branch structure is in a dormant state. After the corresponding target branch structure is determined, the routing unit activates the processing parameter corresponding to the target branch structure based on the activation function, so that the operation unit of the target branch structure performs inference processing on the target input feature information based on the corresponding processing function and the activated processing parameter, obtains output information, and continues to be in a dormant state after the target branch structure completes the inference, thereby reducing the amount of calculation.

[0123] In this way, for the first layer to the Nth layer, the above method is used to determine the target branch structure layer by layer, and finally one or more sparse calculation paths are formed.

[0124] Further, when the last layer of the network layer completes the inference processing on the target input information based on the determined target branch structure to obtain output information, the output information is output to the user.

[0125] In another implementation of the present application, during the inference stage, the initial input information can also be processed based on the pre-cached common input mode and the corresponding calculation path to reduce the routing calculation delay. That is, an inference strategy can also be pre-configured, which is a corresponding relationship between the input mode and the inference path. According to the input information, the corresponding input mode is determined, and the corresponding inference path is determined according to the corresponding relationship between the input mode and the inference path, to obtain the target branch structure corresponding to each network layer, so as to implement the inference processing according to the target branch structure corresponding to the inference path, without branch structure selection calculation, thereby reducing the routing calculation delay.

[0126] The inference method based on the neural network model provided by the implementation of the application is provided. The network of the neural network model is composed of multiple network layers and corresponding routing units. Each network layer includes a plurality of parallel parameter branches. In the calculation process of each layer, only part of the branch structure is selected to participate in the calculation, that is, only part of the parameters of each layer participates in the calculation, instead of using all the parameters of the layer, which can significantly reduce the FLOPs (floating point operation times) and improve the inference speed. Moreover, only the necessary parameters are used for inference, which reduces unnecessary parameter activation, and accurate inference can be realized based on the effective parameters, avoids the interference of unnecessary parameters, improves the accuracy of the inference result, effectively reduces the operation energy consumption, and is particularly suitable for embedded devices, mobile terminals and low-power inference scenarios. Further, the selection of the branch structure is based on the similarity of the input feature information. Different input feature information may pass through different paths in the network. Similar input feature information will select similar branch paths, so as to form a sparse activation path in the entire network. The high consistency of such a path can guarantee the performance of the neural network model. Further, the way of selecting the branch structure to form the inference path according to the input feature information enables the neural network model to automatically use different data modes, and improves the pertinence of the calculation. Moreover, the neural network model of the application can be applied to different application scenarios, can be combined with existing convolutional networks, Transformer structures and the like, is not dependent on a specific model architecture, and has stronger scalability. The sparse ratio of the neural network model can be flexibly adjusted by determining the number of branch structures and selecting the branch structure (for example, TOP-K ratio adjustment), so as to balance the performance and speed in different hardware environments.

[0127] The implementation of the application also provides a computing device. Referring to Figure 7 , Figure 7 Fig. 1 shows a structural schematic diagram of the computing device provided by the embodiment of the application. As shown in Figure 7 , the computing device 10 includes a bus 102, a processor 104, a memory 106 and a communication interface 108. The processor 104, the memory 106 and the communication interface 108 communicate through the bus 102. The computing device 10 can be a server, for example, a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device 10 can also be a terminal device such as a desktop computer, a notebook computer or a smart phone. It should be understood that the number of processors and memories in the computing device 10 is not limited by the application.

[0128] The bus 102 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. The bus 102 can include a path for transmitting information between various components (e.g., the memory 106, the processor 104, the communication interface 108) of the computing device 10.

[0129] The processor 104 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0130] The memory 106 can include a volatile memory, such as a random access memory (RAM). The processor 104 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0131] The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the inference method based on the neural network model. That is, the memory 106 stores instructions for executing the inference method based on the neural network model. Alternatively, the memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the neural network model training method. That is, the memory 106 stores instructions for executing the neural network model training method.

[0132] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card, a transceiver, etc., to implement communication between the computing device 10 and other devices or communication networks.

[0133] The embodiments of the present application also provide a computing device cluster. As shown in Figure 8 The computing device cluster includes at least one computing device 10.

[0134] The memory 106 in one or more of the computing devices 10 in the computing device cluster can store the same instructions for performing the neural network model based inference method and / or the neural network model training.

[0135] In some possible implementation, the memory 106 in one or more of the computing devices 10 in the computing device cluster can also respectively store partial instructions for performing the neural network model based inference method. In other words, the combination of one or more of the computing devices 10 can collectively perform the instructions for performing the neural network model based inference method.

[0136] For example, different branch structures of different network layers of the neural network model can be allocated to different computing devices 10 to implement parallel computing based on the plurality of computing devices 10 to implement the neural network model based inference method provided in the present application.

[0137] The memory 106 in different computing devices 10 in the computing device cluster can store different instructions respectively for performing partial functions of the neural network model based inference method.

[0138] Of course, the memory 106 in one or more of the computing devices 10 in the computing device cluster can store the same instructions for performing the neural network model training method.

[0139] In some possible implementation, the memory 106 in one or more of the computing devices 10 in the computing device cluster can also respectively store partial instructions for performing the neural network model training method.

[0140] The memory 106 in different computing devices 10 in the computing device cluster can store different instructions respectively for performing partial functions of the neural network model training method.

[0141] In some possible implementation, one or more of the computing devices in the computing device cluster can be connected through a network. The network can be a wide area network, a local area network, or the like.

[0142] The embodiments of the present application also provide a chip for running instructions, which is used to perform part of the technical solutions of the neural network model based inference method and / or the neural network model training method in the above embodiments.

[0143] The embodiments of the present application also provide a computer readable storage medium, which stores computer instructions, when the computer instructions are run on a processor of a computer, the processor of the computer performs part of the technical solutions of the neural network model based inference method and / or the neural network model training method in the above embodiments.

[0144] In some possible implementation manners, various aspects of the method provided by the present application can also be implemented in the form of a program product, which includes program codes for causing a processor of a computer to execute the steps in the method described above according to various exemplary implementation manners of the present application, for example, a computer cluster can execute the inference method and / or the neural network model training method based on the neural network model described in the embodiments of the present application.

[0145] The program product can adopt any combination of one or more readable media. The readable medium can be a readable data medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CDROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0146] The implementation manners of the present application also provide a computer program product, which includes a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and the at least one processor can implement part of the technical solutions of the inference method and / or the neural network model training method based on the neural network model in the above embodiments when executing the computer program.

[0147] It should be noted that in addition to the implementation manners of the present application described in the above specific embodiments, other advantages and effects of the present application can be easily understood by those skilled in the art from the content disclosed in the present description. Although the description of the present application is introduced in combination with the preferred embodiments, this does not mean that the features of the present application are limited to the implementation manners. On the contrary, the purpose of introducing the present application in combination with the implementation manners is to cover other options or modifications that can be extended from the present application. In order to provide a deep understanding of the present application, many specific details are included in the above description, and the present application can also be implemented without using these details. In addition, in order to avoid confusion or obscure the focus of the present application, some specific details will be omitted in the description. It should be noted that the embodiments and features in the embodiments in the present application can be combined with each other without conflict.

[0148] It should be noted that in this specification, similar reference numbers and letters indicate similar items in the figures, and thus, once an item is defined in one figure, it should not require further defining or explaining in subsequent figures.

[0149] It should be noted that in the drawings, some structural or methodological features can be shown in particular arrangements and / or orders. However, it should be understood that such particular arrangements and / or orders can not be required. Rather, in some embodiments, the features can be arranged differently than shown in the illustrative figures. Also, the inclusion of a structural or methodological feature in a particular figure is not meant to imply that such feature is required in all embodiments, and in some embodiments, the features can not be included or can be combined with other features.

[0150] While the application has been illustrated and described in detail in the drawings and foregoing description, the same is to be considered as illustrative and not restrictive in character, since the application includes all modifications and equivalents falling within the spirit and scope of the application.

Claims

1. A reasoning method based on a neural network model, characterized in that, The neural network model includes N network layers and routing units corresponding to each network layer. Each network layer includes at least one branch structure, and each branch structure has corresponding processing functions and processing parameters. The method includes: Initial input information is determined and input into the neural network model. The routing units corresponding to each network layer of the neural network model sequentially determine the similarity between the target input information and historical input information input to the routing unit. If historical input information satisfying a preset similarity condition exists, it is used as the target historical input information for the corresponding network layer. The branch structure that performs reasoning processing on the target historical input information is used as the target branch structure for each network layer. If no historical input information satisfying the preset similarity condition exists, the matching information between the target input information and each branch structure of the corresponding network layer is determined, and then... The matching information uses the branch structure that satisfies the target matching condition as the target branch structure corresponding to each network layer, and inputs the target input information into the target branch structure corresponding to each network layer. The target branch structure corresponding to each network layer performs reasoning processing on the target input information according to the processing function and processing parameters corresponding to the target branch structure to obtain the corresponding output information. The target input information input to the routing unit corresponding to the first network layer is the initial input information, the target input information input to the routing unit corresponding to the kth network layer is the output information of the (k-1)th network layer, 1 < k ≤ N, and N is greater than or equal to 2.

2. The reasoning method based on a neural network model according to claim 1, characterized in that, The routing unit includes a lightweight neural network. The routing unit determines the matching information between the target input information and the branch structures corresponding to each branch structure in the network layer. Based on the matching information, it selects the branch structure that satisfies the target matching condition as the target branch structure, including: Based on the lightweight neural network, the activation scores or probability distributions of each branch structure corresponding to the target input information are determined and used as the matching information; Based on the TOP-K selection method, the branch structure that meets the target matching conditions is selected as the target branch structure according to the matching information.

3. The reasoning method based on a neural network model according to claim 1, characterized in that, The routing unit determines the matching information between the target input information and each branch structure included in the corresponding network layer, and selects the branch structure that meets the preset matching conditions as the target branch structure according to the matching information, including: Based on the similarity metric function, the similarity between the target input information and each of the branch structures is determined, and this is used as the matching information. Based on the TOP-K selection method, the branch structure that meets the target matching conditions is selected as the target branch structure according to the matching information.

4. The reasoning method based on a neural network model according to claim 1, characterized in that, The routing unit determines the matching information between the target input information and each of the branch structures included in the corresponding network layer, and selects the branch structure that satisfies the target matching condition as the target branch structure based on the matching information, including: Based on the hash function, the mapping result of the target input information corresponding to each of the branch structures is determined and used as the matching information; Based on the matching information, the branch structure that satisfies the target matching condition is taken as the target branch structure.

5. The reasoning method based on a neural network model according to any one of claims 1-4, characterized in that, After determining the target branch structure, the method further includes: The target branch structure is activated so that it performs inference processing on the target input information according to the processing function and processing parameters corresponding to the target branch structure; wherein, each branch structure is arranged in parallel, and the processing function corresponding to each branch structure is the same but the processing parameters are different, or both the processing function and the processing parameters are different.

6. A method for training a neural network model, characterized in that, The method includes: An initial neural network model is determined, which includes N network layers and routing units corresponding to each network layer. Each network layer includes at least one branch structure, and each branch structure has a corresponding processing function and processing parameters. A training sample set and a training validation set are determined. Based on a soft selection mechanism, forward propagation training and backward propagation training are performed on each branch structure and the corresponding routing unit of each layer of the initial neural network model according to the training sample set to obtain a first neural network model. The training result of the first neural network model is determined based on the training and validation set. When the training result meets the preset training conditions, the first neural network model is optimized for inference based on the training sample set using a hard selection mechanism to obtain a target neural network model. The target neural network model is used to implement the inference method based on the neural network model as described in any one of claims 1-5.

7. The neural network model training method according to claim 6, characterized in that, Based on a soft selection mechanism, forward propagation training and backward propagation training are performed on each branch structure of each layer of the initial neural network model according to the training sample set, including: Determine the similarity between the training samples included in the training sample set; Based on the soft selection mechanism, the path consistency regularization method is adopted to select the training samples whose similarity meets the target similarity condition for the same branch structure and perform forward propagation training and backward propagation training.

8. A computing device cluster, characterized in that, The system includes at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the inference method based on a neural network model as described in any one of claims 1-5, or to cause the computing device cluster to perform the neural network model training method as described in claim 6 or 7.

Images