Hypernetwork training methods, devices, equipment, media and program products

Abstract

This application discloses a supernetwork training method, apparatus, device, medium, and program product, belonging to the field of machine learning technology. The method includes: performing a first-stage training on a supernetwork using a first sample set. The supernetwork includes M subnetworks, which share network parameters with the supernetwork, where M is an integer greater than 1; after the first-stage training, extracting N first subnetworks from the M subnetworks, where N is an integer greater than 1 and less than M; performing cluster analysis on the N first subnetworks to obtain K central subnetworks located at K cluster centers, where K is a positive integer less than or equal to N; and performing a second-stage training on the K central subnetworks using a second sample set until the error of the supernetwork during the second-stage training converges, resulting in a trained supernetwork. This method can improve the error convergence speed of the supernetwork.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a method, apparatus, device, medium, and program product for training hypernetworks. Background Technology

[0002] Neural Architecture Search (NAS) is a technique for automatically designing neural networks.

[0003] The principle of classic NAS is to give a search space containing multiple candidate neural networks with different structures, perform a network search in the search space according to a certain search strategy, evaluate the performance of the candidate neural networks found, and finally search for the neural network with the optimal structure from multiple candidate neural networks.

[0004] During the execution of NAS, each candidate neural network requires network training and performance evaluation, thus the entire search process consumes a large amount of computing resources. Summary of the Invention

[0005] This application provides a method, apparatus, device, medium, and program product for training hypernetworks. The technical solution is as follows:

[0006] According to one aspect of this application, a hypernetwork training method is provided, the method comprising:

[0007] The first sample set is used to train the supernetwork in the first stage. The supernetwork includes M subnetworks. The subnetworks share network parameters with the supernetwork. M is an integer greater than 1.

[0008] After the first stage of training is completed, N first subnetworks are extracted from the M subnetworks, where N is an integer greater than 1 and less than M;

[0009] Cluster analysis is performed on the N first sub-networks to obtain K central sub-networks located at K cluster centers, where K is a positive integer less than or equal to N;

[0010] The second sample set is used to perform a second stage of training on the K central sub-networks until the error of the supernetwork during the second stage of training converges, thus obtaining the trained supernetwork.

[0011] According to another aspect of this application, a hypernetwork training apparatus is provided, the apparatus comprising:

[0012] The training module is used to perform a first-stage training on the supernetwork using a first sample set. The supernetwork includes M subnetworks, and the subnetworks share network parameters with the supernetwork. M is an integer greater than 1.

[0013] The sampling module is used to extract N first sub-networks from the M sub-networks after the first stage of training is completed, where N is an integer greater than 1 and less than M;

[0014] The clustering module is used to perform clustering analysis on the N first sub-networks to obtain K central sub-networks located at K cluster centers, where K is a positive integer less than or equal to N;

[0015] The training module is used to perform a second-stage training on the K central sub-networks using a second sample set until the error of the supernetwork during the second-stage training converges, thus obtaining the trained supernetwork.

[0016] According to another aspect of this application, a computer device is provided, the computer device including a processor and a memory connected to the processor, the memory storing program instructions, wherein the processor executes the program instructions to implement the hypernetic network training method provided in various aspects of this application.

[0017] According to another aspect of this application, a computer-readable storage medium is provided, wherein program instructions are stored therein, which, when executed by a processor, implement the hypernetic network training method provided in various aspects of this application.

[0018] According to another aspect of this application, a computer program product (or computer program) is provided, the computer program product (or computer program) including computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the hypernetic network training method as provided in various aspects of this application.

[0019] According to another aspect of this application, a chip is provided, the chip including programmable logic circuitry and / or program instructions, which, when the chip is running, are used to implement the hypernetwork training methods provided in various aspects of this application.

[0020] The beneficial effects of the technical solutions provided in this application embodiment may include:

[0021] In the above-mentioned hypernetwork training method, the hypernetwork is first trained in the first stage. After the first stage of training is completed, some subnetworks are selected and clustered to obtain K central subnetworks located at K cluster centers. Since the hypernetwork and subnetworks share network parameters, the second stage of training will be carried out on the K central subnetworks. That is, after clustering, the central subnetworks are trained in a targeted manner, instead of training all subnetworks in the hypernetwork. This is beneficial to accelerate the convergence of errors during hypernetwork training and can improve the training efficiency of the hypernetwork. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating a hypernetwork training method provided in an exemplary embodiment of this application is shown;

[0024] Figure 2 A schematic diagram of sub-network sampling provided in an exemplary embodiment of this application is shown;

[0025] Figure 3 A schematic diagram of sub-network sampling provided by another exemplary embodiment of this application is shown;

[0026] Figure 4 A flowchart of a hypernetwork training method provided in another exemplary embodiment of this application is shown;

[0027] Figure 5 A schematic diagram of sub-network clustering provided in an exemplary embodiment of this application is shown;

[0028] Figure 6 A flowchart illustrating an exemplary embodiment of the error calculation method provided in this application is shown.

[0029] Figure 7 A flowchart of a hypernetwork training method provided in another exemplary embodiment of this application is shown;

[0030] Figure 8 A block diagram of a hypernetwork training apparatus provided in an exemplary embodiment of this application is shown;

[0031] Figure 9 A schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0033] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this application, it should be noted that, unless otherwise explicitly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0035] First, let me introduce some of the terms used in this application:

[0036] A supernet is a network composed of multiple neural network structures. In a supernet, each neural network structure is also called a subnet.

[0037] For example, during the execution of NAS, multiple neural network structures in the search space need to be trained and tested sequentially. The entire search process consumes a large amount of computing resources. Therefore, a one-shot NAS technique was proposed. In One-Shot NAS, a unified network structure is constructed, within which multiple neural network structures can be searched, essentially combining multiple neural network structures in the search space into a larger network structure. The technical implementation includes four steps: supernetwork training, subnetwork search, subnetwork fine-tuning, and subnetwork performance testing. The network parameters in the supernetwork are shared, meaning the supernetwork and subnetworks share network parameters. During supernetwork training, each round of training requires searching for a subnetwork from the supernetwork for training until the supernetwork's error converges, completing the training of the supernetwork.

[0038] Generally, the search space of a hypernetwork is very large. In particular, the number of subnetworks contained in the search space can increase exponentially with the number of network layers. Training a large number of subnetworks with different structures leads to a very slow error convergence speed for the hypernetwork. Therefore, this application provides a hypernetwork training method, the detailed implementation of which is shown in the following embodiment.

[0039] Figure 1 This is a flowchart of a hypernetwork training method provided in an exemplary embodiment of this application. The method is applied in a computer device, which may be a terminal or a server, and includes:

[0040] Step 110: The first sample set is used to train the supernetwork in the first stage. The supernetwork includes M subnetworks. The subnetworks and the supernetwork share network parameters. M is an integer greater than 1.

[0041] For example, a sample set is stored in a computer device; or, a sample set is stored in a database. The sample set includes a first sample set; the computer device retrieves the first sample set from its own memory or the database; and the first sample set is used to perform a first stage of training on the hypernetwork.

[0042] During the first phase of training, for each round of training, the computer device samples a third subnetwork from the supernetwork; inputs the training samples from the first sample set into the third subnetwork to obtain the second output result; calculates the first error between the second output result and the actual result; and adjusts the network parameters in the third subnetwork based on the first error, that is, adjusts the network parameters of the supernetwork.

[0043] For example, the hypernetwork supports subnetwork search for image recognition; correspondingly, the training samples in the first sample set include image samples labeled with standard results; the computer device inputs the above image samples into the third subnetwork and outputs the recognition result of the image samples (i.e., the second output result); calculates the first error between the recognition result and the standard result (i.e., the actual result); and adjusts the network parameters in the third subnetwork based on the first error.

[0044] The computer equipment uses the first sample set to perform Q rounds of training on the supernetwork, completing the first stage of training of the supernetwork; where the value of Q is preset by the designer.

[0045] Alternatively, during the training of the supernetwork, the computer device monitors the convergence of the supernetwork's error; if the supernetwork's error is less than a first error threshold, the first stage of training of the supernetwork ends. This first error threshold is preset by the designers. For example, after calculating the first error, if the supernetwork's first error is less than the first error threshold, the computer device stops adjusting the supernetwork's parameters, completing the first stage of training.

[0046] For example, the sample set mentioned above includes at least one of image samples, video samples, speech samples, and text samples; that is, the hypernetwork in this application supports neural network structure search for images, videos, speech, text, etc.

[0047] For example, for the search of the third subnetwork, the computer device can use any of the following search methods: random search, Bayesian optimization algorithm, evolutionary algorithm, reinforcement learning, gradient-based algorithm.

[0048] Step 120: After the first stage of training is completed, extract N first subnetworks from the M subnetworks, where N is an integer greater than 1 and less than M.

[0049] For example, for the search of the first subnetwork, the computer device can employ any of the following search methods: random search, Bayesian optimization algorithm, evolutionary algorithm, reinforcement learning, or gradient-based algorithm. For instance, after the first stage of training, the computer device randomly searches for N first subnetworks from M subnetworks. The search methods for extracting the first and third subnetworks can be the same or different.

[0050] A scaling factor list is provided, which includes at least two scaling factors. During the process of searching for a first sub-network from the supernetwork, the computer device selects at least one scaling factor as the scaling ratio for acquiring each layer of the first sub-network. The scaling factor indicates the width ratio between the sub-network and the corresponding layers of the supernetwork. The first sub-network is sampled from the supernetwork based on at least one scaling factor. For example, the at least one scaling factor may be randomly selected from the scaling factor list.

[0051] For example, such as Figure 2 As shown, in an L-layer hypernetwork, each layer has a width of 4, where L is an integer greater than 1. When sampling subnetworks in this hypernetwork, the scaling factor list used is {0.25, 0.5, 0.75, 1.0}. The computer device can use two or more scaling factors from the scaling factor list to sample each layer of the first subnetwork, obtaining subnetworks 1 to C, where C is greater than 1 and less than or equal to L. 4 The integers. Specifically, for the search of subnetwork 1, the computer device uses 1.0 to collect layer 1 with a width of 4, 0.75 to collect layer 2 with a width of 3, and 0.5 to collect layer 3 with a width of 2; for the search of subnetwork C, the computer device uses 0.5 to collect layer 1 with a width of 2, and 0.75 to collect layers 2 and 3 with a width of 3.

[0052] Optionally, the computer device employs a uniform sampling method to extract N first subnetworks from M subnetworks. Uniform sampling refers to sampling each layer of the neural network structure using the same shrink ratio. For example, ... Figure 3 As shown, each layer in an L-layer hypernetwork has a width of 4. When sampling subnetworks in this hypernetwork, the scaling factor list used is {0.25, 0.5, 0.75, 1.0}. The computer device can use any scaling factor from the list to sample each layer in the first subnetwork, obtaining subnetworks 1 to 4. Specifically, for the search of subnetwork 1, the computer device uses a scaling factor of 0.75 to sample layers 1 to 3 with a width of 3; for the search of subnetwork 4, the computer device uses a scaling factor of 0.5 to sample layers 1 to 3 with a width of 2. For example, the aforementioned scaling factor can be randomly selected from the scaling factor list.

[0053] It should also be noted that during the first stage of training, the computer equipment can also use a uniform sampling method to extract subnetworks from the supernetwork for training.

[0054] Step 130: Perform cluster analysis on the N first subnetworks to obtain K central subnetworks located at K cluster centers, where K is a positive integer less than or equal to N.

[0055] The computer equipment uses a clustering algorithm to perform cluster analysis on N first sub-networks to obtain K cluster sets. Each cluster set includes a sub-network located at the cluster center, i.e., the central sub-network. K central sub-networks are obtained from the K cluster sets.

[0056] For example, the clustering algorithm described above can be any of the following: K-means algorithm, binary K-means algorithm, K-Nearest Neighbor (KNN) classification algorithm, or Gaussian Mixed Model (GMM). For instance, a computer device might use the K-means algorithm to calculate the K central subnetworks located at the K cluster centers out of N first subnetworks; where the value of K is preset by the designer.

[0057] For example, the computer device uses a clustering algorithm to perform clustering analysis on N first sub-networks based on the similarity between the sub-networks. For instance, the computer device uses a clustering algorithm to perform clustering analysis on N first sub-networks based on the structural similarity between the sub-networks.

[0058] Step 140: Use the second sample set to perform the second stage training on the K central subnetworks until the error of the supernetwork during the second stage training converges, and obtain the trained supernetwork.

[0059] For example, the above sample set includes a second sample set; the computer device obtains the second sample set from its own memory or database; the second sample set is used to perform a second-stage training on the K central sub-networks until the error of the supernetwork during the second-stage training converges, and the trained supernetwork is obtained.

[0060] For example, the computer device uses a second sample set to iteratively train the K central subnetworks until the error of the supernetwork converges during the second stage of training, thus obtaining the trained supernetwork. For instance, there are three central subnetworks: subnetwork 01, subnetwork 02, and subnetwork 03. The computer device uses training samples from the second sample set to train subnetwork 01, subnetwork 02, and subnetwork 03 sequentially for one round, and then trains subnetwork 01, subnetwork 02, and subnetwork 03 sequentially for another round, and so on, until the error of the supernetwork converges, thus obtaining the trained supernetwork.

[0061] During the second phase of training, for each round of training, the computer device inputs the training samples from the second sample set into the central subnetwork to obtain the third output result; calculates the third error between the third output result and the actual result; and adjusts the network parameters in the central subnetwork based on the third error, that is, adjusts the network parameters of the supernetwork.

[0062] During the training of a supernetwork, in order to characterize the network structure of a subnetwork, the computer device encodes the list of network structure information of the subnetwork using one-hot encoding to obtain the encoded identifier of the subnetwork, which is used to uniquely identify the subnetwork.

[0063] It should be noted that the structures of different sub-networks in the embodiments of this application are different, and / or the shrinkage rates used in different sub-networks may be the same or different.

[0064] In summary, the hypernetwork training method provided in this embodiment first performs a first-stage training on the hypernetwork. After completing the first-stage training, a portion of the subnetworks are selected, and cluster analysis is performed on these subnetworks to obtain K central subnetworks located at K cluster centers. Since the hypernetwork and subnetworks share network parameters, a second-stage training will be performed on the K central subnetworks. That is, after clustering, network training is performed specifically on the central subnetworks, rather than training all subnetworks in the hypernetwork. This is beneficial for accelerating the convergence of errors during hypernetwork training and can improve the training efficiency of the hypernetwork.

[0065] Secondly, during the entire training process of the supernetwork, the computer equipment uses a uniform sampling method to extract subnetworks from the supernetwork, so that the width scaling ratio of each layer in each sampled subnetwork is consistent, reducing the diversity of the subnetwork structure during training. This is more conducive to the rapid convergence of errors during supernetwork training.

[0066] To accelerate error convergence and improve the accuracy of the supernetwork, the computer can select a central subnetwork from the high-performing subnetworks for the second stage of training. For example, the computer uses a uniform sampling method to extract N first subnetworks that meet the first performance requirements from M subnetworks, such as... Figure 4 As shown, it can be Figure 2 Replace step 120 with steps 122 to 126 to achieve this step, as shown below:

[0067] Step 122: After the first stage of training is completed, a uniform sampling method is used to extract G second subnetworks from M subnetworks, where G is greater than or equal to N and less than M.

[0068] For example, the computer device can employ any of the following search methods to search for the second subnetwork: random search, Bayesian optimization algorithm, evolutionary algorithm, reinforcement learning, or gradient-based algorithm. For instance, after the first stage of training, the computer device randomly searches for G second subnetworks from the M subnetworks. The search methods used to extract the second and third subnetworks can be the same or different.

[0069] A scaling factor list is provided, which includes at least two scaling factors. During the process of searching for a second subnetwork from the supernetwork, the computer device selects a scaling factor as the scaling ratio for sampling each layer of the second subnetwork. The second subnetwork is then sampled from the supernetwork based on this scaling factor. For example, one of the scaling factors can be randomly selected from the scaling factor list.

[0070] Step 124: Use the first test sample to perform performance testing on the second sub-network, and obtain G performance test results corresponding to G second sub-networks.

[0071] The computer device stores a first test sample; or, the database stores the first test sample. The computer device retrieves the first test sample from its own memory or the database; it then uses the first test sample to perform performance testing on the second sub-network, obtaining G performance test results corresponding to the G second sub-networks. Here, performance testing, also known as performance evaluation, is used to assess the generalization ability of the trained neural network structure. The performance test results of the second sub-networks are used to indicate the generalization ability of the second sub-networks.

[0072] For example, the performance test results include any one of the following data: error rate; accuracy; precision (P) and recall (R); metric F1, where F1 = (2 × P × R) / (P + R), where "×" indicates multiplication and " / " indicates division; and the AUC (Area Under Curve) of the ROC (Receiver Operating Characteristic) curve. That is, the computer device can use error rate and accuracy to evaluate the generalization ability of the subnetwork; or it can evaluate the generalization ability of the subnetwork from the perspective of precision and recall; or it can use the ROC curve to evaluate the generalization ability of the subnetwork. In practice, the evaluation method used can be designed according to the task requirements.

[0073] Step 126: Based on the G performance test results, determine N first sub-networks that meet the first performance requirements from the G second sub-networks.

[0074] Optionally, the first performance requirement includes the subnetwork's generalization ability being higher than a first generalization ability. For example, if the performance test result includes accuracy, the first performance requirement includes the subnetwork's accuracy being higher than a first accuracy threshold; or if the performance test result includes error rate, the first performance requirement includes the subnetwork's error rate being lower than a first error rate threshold; or if the performance test result includes F1, the first performance requirement includes the subnetwork's F1 score being greater than a first threshold. Here, the first generalization ability is a pre-set capability threshold.

[0075] For example, a computer device determines N first subnetworks from G second subnetworks whose generalization ability is higher than that of a first generalization ability, based on G performance test results. For instance, the computer device determines N first subnetworks from G second subnetworks whose accuracy is higher than a first accuracy threshold, based on G precision values; or, for example, the computer device determines N first subnetworks from G second subnetworks whose F1 score is higher than a first threshold, based on G F1 scores.

[0076] Optionally, the first performance requirement includes that the generalization ability of the subnetwork is among the top N in descending order. For example, the computer device sorts G F1 scores from largest to smallest, determines the top N F1 scores, and identifies the N second subnetworks corresponding to the top N F1 scores as N first subnetworks; or, for another example, the computer device sorts G error rates from smallest to largest, determines the top N error rates, and identifies the N second subnetworks corresponding to the top N error rates as N first subnetworks.

[0077] In some embodiments, to reduce the computational burden of performance evaluation, the test errors of subnetworks can be used to characterize the generalization ability of subnetworks. In this scenario, for the implementation of steps 124 to 126 above, the computer device can use the first test sample to test the error between the output result of the second subnetwork and the actual result, and obtain G test errors corresponding to G second subnetworks; according to the G test errors in ascending order, the first N second subnetworks corresponding to the first N test errors are selected to obtain N first subnetworks that meet the first performance requirements.

[0078] Alternatively, after obtaining G test errors, the computer device can also determine N test errors that are less than the second error threshold from the G test errors; and determine the N second sub-networks corresponding to the N test errors as N first sub-networks that meet the first performance requirements.

[0079] That is, the first performance requirement includes the subnetwork's test error being among the top N in ascending order. Alternatively, the first performance requirement includes the subnetwork's test error being less than a second error threshold. The second error threshold is preset. For example, the first error threshold can be less than or equal to the second error threshold.

[0080] For example, such as Figure 5 As shown, if K is 4, after obtaining N first sub-networks with good performance, the computer device uses the K-means algorithm to cluster the N first sub-networks, resulting in 4 clusters: sub-network cluster 1, sub-network cluster 2, sub-network cluster 3 and sub-network cluster 4; the sub-network located at the center of the cluster is the central sub-network.

[0081] In summary, the hypernetwork training method provided in this embodiment, during the clustering analysis process, extracts N high-performing first subnetworks from the hypernetwork by setting a first performance requirement, filters out low-performing subnetworks, and then clusters K central subnetworks from the N first subnetworks. This ensures that the subnetworks in the second stage of training all have good performance. This avoids the interference of low-performing subnetworks with the weight adjustment of high-performing subnetworks, which would reduce the performance of the trained hypernetwork and thus improve its performance. Furthermore, this method can also reduce the search space of subnetworks, training only the most representative subnetworks and improving the error convergence speed during the hypernetwork training process.

[0082] Secondly, when evaluating the performance of the subnetwork in the above method, the test error of the subnetwork can be used. Compared with performance evaluation methods such as accuracy, error rate, and F1, this can greatly reduce the amount of computation, thereby improving the performance evaluation time of the subnetwork, reducing the overall training time of the supernetwork, and improving the training efficiency of the supernetwork.

[0083] In order to improve the efficiency of the first-stage training process, the third sub-network can also be trained based on knowledge distillation. For example, step 110 can be implemented as follows:

[0084] Step 110(a): During the first phase of training, obtain the baseline network.

[0085] The aforementioned benchmark network is a pre-trained neural network structure, and both the aforementioned benchmark network and the aforementioned supernetwork are neural network structures designed for the same task requirements.

[0086] Optionally, the aforementioned benchmark network is a neural network structure independent of the supernetwork. For example, the benchmark network is a neural network structure trained until the error converges; or, the benchmark network is a neural network structure trained until the error is less than a third error threshold. This benchmark network can be trained by a computer device; or it can be trained by another device and then transferred to that computer device for use, where other devices refer to computer devices other than the aforementioned computer device.

[0087] Optionally, the aforementioned baseline network is a sub-network within the supernetwork. The aforementioned baseline network is a neural network structure trained until the error is less than a third error threshold. For example, the first stage of training the supernetwork by the computer device can be divided into two sub-stages; in the first sub-stage, the computer device trains the supernetwork using a subset of samples from a first sample set; if a sub-network with an error less than the third error threshold exists within the supernetwork, that sub-network is used as the baseline network; in the second sub-stage, steps 110(b) to 110(f) are performed.

[0088] For example, in the first sub-stage, the computer device selects a third sub-network from the supernetwork; inputs training samples from the first sample set into the third sub-network to obtain a second output result; calculates a first error between the second output result and the actual result; and if the first error is less than a third error threshold, uses this third sub-network as the baseline network. Alternatively, after calculating the first error, the computer device adjusts the network parameters in the third sub-network based on the first error; tests the error of this third sub-network using a third test sample; if the error obtained from the test is less than the third error threshold, uses this third sub-network as the baseline network; and then executes steps 110(b) to 110(f) to extract another third sub-network from the supernetwork for a new round of training.

[0089] The aforementioned third error threshold is preset. For example, when the aforementioned reference network is a neural network structure independent of the supernetwork, the magnitude relationship between the third error threshold and the first error threshold, as well as the second error threshold, is not limited; when the aforementioned reference network is a subnetwork in the supernetwork, the aforementioned third error threshold may be greater than or equal to the aforementioned first error threshold; and / or, the aforementioned third error threshold may also be greater than or equal to the aforementioned second error threshold.

[0090] Step 110(b): For each round of training, select a third subnetwork from the M subnetworks.

[0091] Step 110(c): Input the training samples from the first sample set into the baseline network to obtain the first output result; and input the training samples into the third sub-network to obtain the second output result.

[0092] For example, the first output result is obtained by the baseline network processing the training samples according to the task requirements; the second output result is obtained by the third sub-network processing the training samples according to the task requirements. In each round of training, the first output result and the second output result are obtained by processing the same training samples according to the same task requirements.

[0093] It should be noted that the calculation order of the first output result and the second output result is not limited in this embodiment. The simultaneous calculation method in step 110(c) is used as an example for illustration.

[0094] Step 110(d): Calculate the first error between the second output result and the actual result.

[0095] In some embodiments, the first error can also be calculated using the output of the first intermediate layer in the third sub-network and the output of the second intermediate layer in the reference network; for example, a computer device calculates the error between the output of the first intermediate layer and the output of the second intermediate layer as the first error; wherein, the second intermediate layer is the intermediate layer in the reference network corresponding to the first intermediate layer.

[0096] Step 110(e): Calculate the second error between the first output result and the second output result.

[0097] It should be noted that the execution order of steps 110(d) and 110(e) is not limited in this embodiment. The example is to illustrate the sequential execution of steps 110(d) and 110(e).

[0098] Step 110(f): Based on the first error and the second error, adjust the network parameters in the third sub-network.

[0099] The third subnetwork shares network parameters with the supernetwork. The computer device adjusts the network parameters in the third subnetwork based on the weighted sum of the first and second errors to obtain the adjusted network parameters, which are the network parameters of the supernetwork.

[0100] The weight corresponding to the first error is preset as the first weight, and the weight corresponding to the second error is preset as the second weight; the computer device multiplies the first error by the first weight to obtain the first product; and multiplies the second error by the second weight to obtain the second product; the first product and the second product are summed to obtain the weighted sum of the first error and the second error, which is the weighted error; the network parameters in the third sub-network are adjusted according to the weighted error; wherein, the sum of the first weight and the second weight is 1.

[0101] Optionally, the values ​​of the first weight and the second weight are preset empirical values.

[0102] Optionally, the first and second weights are learned during network training. That is, the first and second weights, as network parameters of the supernetwork, are learned during the training of the supernetwork. For example, the network parameters of the third subnetwork include the first and second weights; during the first round of training, the computer uses the initial values ​​of the first and second weights; during the (i+1)th round of training, the computer uses the first and second weights adjusted during the i-th round of training.

[0103] For example, if the network parameters in the third sub-network are some of the network parameters in the supernetwork, then after obtaining the adjusted network parameters, the computer device updates some of the network parameters of the supernetwork based on the adjusted network parameters.

[0104] As mentioned above, the training of the third subnetwork can be carried out using knowledge distillation. In the process of knowledge distillation, the aforementioned baseline network can be called the teacher network, and the third subnetwork can be called the student network.

[0105] In summary, the supernetwork training method provided in this embodiment obtains a baseline network during the first stage of training to participate in the training of subnetworks within the supernetwork. Knowledge distillation is used to train the subnetworks within the supernetwork, accelerating their learning speed and thus improving the convergence speed of the supernetwork. Furthermore, this method can select a high-performance baseline network using a third error threshold, ensuring both the learning speed and accuracy of the subnetworks within the supernetwork.

[0106] In some embodiments, when steps 110(d) and 110(e) are executed sequentially, the values ​​of the first weight and the second weight are both greater than 0. Before calculating the second error between the first output result and the second output result, the computer device calculates the third error between the first output result and the actual result. If the difference between the third error and the first error is less than or equal to the difference threshold, the value of the first weight is reset to 1 and the value of the second weight is reset to 0. Then, steps 110(e) to 110(f) are executed.

[0107] For example, such as Figure 6 As shown, the weighted error is calculated during network training using knowledge distillation in computer equipment.

[0108] 210. Input the training samples from the first sample set into subnetwork 10 and teacher network 20 respectively. Subnetwork 10 is also the third subnetwork.

[0109] 220, the training samples are processed through subnetwork 10 to output Y. sub Y sub That is, the second output result.

[0110] 230, calculate Y sub and (i.e., the actual result) This is the first error.

[0111] 240, the training samples are processed through teacher network 20, and the output Y is obtained. teacher Y teacher That is, the first output result.

[0112] 250, calculate Y teacher and (i.e., the actual result) This is the third error.

[0113] 260. Determine if Loss0 is less than or equal to α; if not, proceed to step 270; if yes, proceed to step 280.

[0114] Where Loss0 is the difference between the third error and the first error, expressed by the formula Loss0 =

[0115]

[0116] 270, Calculate With Loss1(Y sub ,Y teacher The weighted sum of ) gives the Loss.

[0117] Where Loss is the weighted error, Loss1(Y) sub ,Y teacher The second error is the difference between the second output and the first output; the loss is expressed by the formula: W0 is the first weight, and W1 is the second weight.

[0118] For example, during initialization, W0 = 0.5 and W1 = 0.5.

[0119] 280, take the first error as the loss.

[0120] Loss is expressed by the formula as follows: That is, W0 = 1, W1 = 0.

[0121] In summary, the supernetwork training method provided in this embodiment, when the difference between the third error and the first error is less than or equal to a difference threshold, will no longer refer to the error of the teacher network when adjusting the network parameters. This can prevent the network from inhibiting the learning of the subnetwork in the later stages of network training, because due to the limitations of the error of the teacher network itself, it may have an inhibitory effect on the learning of the subnetwork in the later stages of network training.

[0122] During the second phase of training, in order to ensure that all subnetworks in the supernetwork have good performance, the computer device can also sample and test whether the performance of the subnetworks meets the second performance requirement. For example, step 140 can be implemented as follows:

[0123] Step 140(a): Use the second sample set to perform the second stage training on the K central subnetworks.

[0124] Step 140(b): If the error of the supernetwork converges, extract D subnetworks to be tested from the M subnetworks, where D is an integer greater than 0 and less than M.

[0125] For example, for searching the subnetwork under test, the computer device can use any of the following search methods: random search, Bayesian optimization algorithm, evolutionary algorithm, reinforcement learning, gradient-based algorithm.

[0126] The subnetwork under test may be searched using the same or different methods as the first and / or third subnetworks. For example, the computer device selects at least one scaling factor from a scaling factor list as the scaling ratio for each layer of the network in the test subnetwork; and samples the test subnetwork from the supernetwork based on the at least one scaling factor. For example, the at least one scaling factor may be randomly selected from the scaling factor list.

[0127] Optionally, the computer device employs a uniform sampling method to extract D subnetworks to be tested from M subnetworks. For example, a scaling factor used in the uniform sampling method can be randomly selected from a list of scaling factors.

[0128] Step 140(c): Use the second test sample to perform performance testing on the D sub-networks under test.

[0129] For example, the computer device can use at least two second test samples to perform performance tests on each subnetwork under test, obtaining D test results for D subnetworks under test. The test results of the test subnetworks are used to indicate the generalization ability of the subnetwork under test.

[0130] The generalization ability described above can be represented using any of the following information: accuracy; precision and recall; metric F1; AUC of the ROC curve. Optionally, the generalization ability can also be represented using the error of the sub-network under test; the computer device uses a second test sample to perform performance testing on the sub-network under test, obtaining the error of the sub-network under test as the test result. For example, if the second test sample includes at least two samples, the mean error of the sub-network under test is used as the test result.

[0131] Step 140(d): If the performance of all D sub-networks to be tested meets the second performance requirement, the supernetwork training is considered complete.

[0132] For example, the second performance requirement includes that the generalization ability of the sub-network under test is higher than the second generalization ability. For instance, if the test result includes accuracy, the second performance requirement includes that the accuracy of the sub-network under test is higher than a second accuracy threshold; or if the test result includes F1, the second performance requirement includes that the F1 score of the sub-network under test is greater than a second threshold; or if the test result includes the error of the sub-network under test, the second performance requirement includes that the error of the sub-network under test is less than a fourth error threshold. Here, the second generalization ability is a pre-set capability threshold. For example, the second generalization ability is higher than the first generalization ability.

[0133] Step 140(e): If at least one of the D sub-networks under test fails to meet the second performance requirement, the K central sub-networks are trained again using the third sample set until the performance of the sub-networks under test that are resampled and tested in the supernetwork meets the second performance requirement.

[0134] For example, the intersection of the second sample set and the third sample set may be non-empty or empty.

[0135] In summary, the supernetwork training method provided in this embodiment performs sampling inspections on the subnetworks within the supernetwork after the supernetwork's error converges, to ensure that all subnetworks in the supernetwork can achieve good performance.

[0136] like Figure 7 The flowchart shown is a supernetwork training method provided in an exemplary embodiment of this application. This flowchart illustrates the overall idea of ​​supernetwork training in this application, and the steps are as follows:

[0137] Step 310, Begin.

[0138] Step 320: Train the baseline network.

[0139] For example, a computer device independently trains a baseline network to obtain a baseline network with converged error. This baseline network can also be called a baseline model. This baseline model can be a neural network structure designed independently of the supernetwork, or it can be a neural network structure extracted from the supernetwork.

[0140] Step 330: Based on the baseline network, train the subnetwork obtained by uniform sampling in the supernetwork.

[0141] For example, the computer device extracts a third subnetwork from the supernetwork using a uniform sampling method, and uses the baseline network as the teacher network to train the third subnetwork, thus completing the first stage of training the supernetwork.

[0142] Step 340: Sample G second subnetworks from the trained supernetwork, filter out N first subnetworks from the G second subnetworks, perform cluster analysis on the N first subnetworks to obtain K central subnetworks located at K cluster centers.

[0143] The computer equipment uses a uniform sampling method to sample G second sub-networks from the supernetwork after the first stage of training; it then filters out N first sub-networks that meet the first performance requirements from the G second sub-networks, that is, it filters out N first sub-networks whose generalization ability is greater than the first generalization ability from the G second sub-networks; and it uses the K-means algorithm to cluster the N first sub-networks to obtain K central sub-networks located at K cluster centers.

[0144] Step 350: Train the K central subnetworks.

[0145] The computer equipment sequentially trains the K central subnetworks and adjusts the network parameters in the supernetwork; the K central subnetworks share network parameters with the supernetwork.

[0146] Step 360: If the error of the supernetwork converges, sample D subnetworks to be tested from the supernetwork.

[0147] Under the condition that the error of the supernetwork converges, the computer device uses a uniform sampling method to sample D subnetworks from the supernetwork as the subnetworks to be tested. The computer device then performs performance tests on these D subnetworks.

[0148] Step 370: Determine whether the performance of the D sub-networks under test meets the second performance requirement; if not, return to step 350; if yes, proceed to step 380.

[0149] In some embodiments, the computer device may further determine whether the ratio of the sub-network under test that meets the second performance requirement to D is greater than the ratio threshold. If not, return to step 350; if yes, execute step 380.

[0150] Step 380, End.

[0151] It should be noted that the detailed implementation of each step in this embodiment can be found in the above embodiments, and will not be repeated here.

[0152] In summary, the hypernetwork training method provided in this embodiment introduces knowledge distillation during the hypernetwork training process and utilizes a high-performance teacher network to guide the training of subnetworks, thereby accelerating the convergence of the hypernetwork.

[0153] Secondly, this method also uses a dynamic loss function. For example, the network's loss function consists of two parts: the true loss value of the sub-network. and the output loss value Loss1(Y) of the subnetwork and the teacher network sub ,Y teacher To prevent the teacher network from inhibiting the training of subnetworks in the later stages of training, when At this point, the loss function weights for the subnetwork output and the teacher network output are reset to 0. This method achieves this by using the teacher network to guide training in the early stages of training, accelerating convergence, and then focusing on training the model itself in the later stages of training, further improving the performance of the supernetwork.

[0154] Furthermore, after completing the initial training of the supernetwork, the subnetworks are clustered, and cluster center subnetworks are trained. For example, some subnetworks are randomly sampled, and those with poor performance are filtered out. Only the high-performing subnetworks are subjected to K-means clustering, and cluster centers are trained. This filtering effectively avoids interference from poor-performing subnetworks on the supernetwork performance, thereby improving the supernetwork's performance. Additionally, this approach reduces the sampling space of the subnetworks, training only the most structurally representative subnetworks, thus improving the training convergence speed.

[0155] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0156] Figure 8 This illustration shows a structural block diagram of a hypernetwork training apparatus provided in an exemplary embodiment of this application. The apparatus can be implemented as all or part of an in-vehicle device through software, hardware, or a combination of both. The apparatus establishes a communication connection with at least one terminal and includes:

[0157] Training module 410 is used to perform a first-stage training on the supernetwork using a first sample set. The supernetwork includes M subnetworks, and the subnetworks share network parameters with the supernetwork. M is an integer greater than 1.

[0158] The sampling module 420 is used to extract N first sub-networks from the M sub-networks after the first stage of training is completed, where N is an integer greater than 1 and less than M;

[0159] Clustering module 430 is used to perform clustering analysis on the N first sub-networks to obtain K central sub-networks located at K cluster centers, where K is a positive integer less than or equal to N;

[0160] The training module 410 is used to perform a second-stage training on the K central sub-networks using a second sample set until the error of the supernetwork during the second-stage training converges, thus obtaining a trained supernetwork.

[0161] In some embodiments, the sampling module 420 is configured to:

[0162] Using a uniform sampling method, N first sub-networks that meet the first performance requirements are extracted from the M sub-networks;

[0163] The uniform sampling method refers to the method of sampling each layer in the neural network structure using the same scaling factor.

[0164] In some embodiments, the sampling module 420 is configured to:

[0165] Using the uniform sampling method, G second sub-networks are extracted from the M sub-networks, where G is greater than or equal to N and less than M;

[0166] The performance of the second sub-network is tested using the first test sample, and G performance test results are obtained for the G second sub-networks.

[0167] Based on the G performance test results, N first sub-networks that meet the first performance requirements are determined from the G second sub-networks.

[0168] In some embodiments, the performance test results include the test error of the second sub-network; the sampling module 420 is configured to:

[0169] The error between the output result of the second sub-network and the actual result is tested using the first test sample, and G test errors corresponding to the G second sub-networks are obtained.

[0170] According to the G test errors in ascending order, select the first N second sub-networks corresponding to the first N test errors to obtain the N first sub-networks that meet the first performance requirements.

[0171] In some embodiments, the training module 410 is configured to:

[0172] During the first phase of training, a baseline network is acquired;

[0173] For each round of training, a third subnetwork is selected from the M subnetworks;

[0174] The training samples from the first sample set are input into the baseline network to obtain a first output result; and the training samples are input into the third sub-network to obtain a second output result;

[0175] Calculate the first error between the second output result and the actual result;

[0176] Calculate the second error between the first output result and the second output result;

[0177] Based on the first error and the second error, the network parameters in the third sub-network are adjusted.

[0178] In some embodiments, the training module 410 is configured to:

[0179] Multiply the first error by the first weight to obtain a first product; and multiply the second error by the second weight to obtain a second product;

[0180] Summing the first product and the second product yields the weighted error;

[0181] The network parameters in the third sub-network are adjusted according to the weighted error; wherein the sum of the first weight and the second weight is 1.

[0182] In some embodiments, the training module 410 is configured to:

[0183] If both the first weight and the second weight are greater than 0, before calculating the second error between the first output result and the second output result, calculate the third error between the first output result and the actual result;

[0184] If the difference between the third error and the first error is less than the difference threshold, the value of the first weight is reset to 1, and the value of the second weight is reset to 0.

[0185] In some embodiments, the training module 410 is configured to:

[0186] The second sample set is used to perform the second stage training on the K central sub-networks;

[0187] If the error of the supernetwork converges, D subnetworks to be tested are extracted from the M subnetworks, where D is an integer greater than 0 and less than M;

[0188] The performance of the D sub-networks under test is performed using a second test sample.

[0189] If the performance of all D sub-networks under test meets the second performance requirement, the training of the supernetwork is considered complete.

[0190] In some embodiments, the training module 410 is used to retrain the K central subnetworks using a third sample set if the performance of at least one of the D subnetworks to be tested does not meet the second performance requirement, until the performance of all the subnetworks to be tested that are resampled in the supernetwork meets the second performance requirement.

[0191] Figure 9 A schematic diagram of a computer device provided in an exemplary embodiment of this application is shown. This computer device may be a device for executing the hypernetwork training method provided in this application. For example, the computer device may be a server or a terminal. Specifically:

[0192] Computer device 1000 includes a central processing unit (CPU) 1001, a system memory 1004 including random access memory (RAM) 1002 and read-only memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the central processing unit 1001. Computer device 1000 also includes a basic input / output system (I / O system) 1006 that facilitates information transfer between various devices within the computer, and a mass storage device 1007 for storing the operating system 1013, application programs 1014, and other program modules 1015.

[0193] The basic input / output system 1006 includes a display 1008 for displaying information and an input device 1009 for user input, such as a mouse or keyboard. Both the display 1008 and the input device 1009 are connected to the central processing unit 1001 via an input / output controller 1010 connected to the system bus 1005. The basic input / output system 1006 may also include the input / output controller 1010 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1010 also provides output to a display screen, printer, or other types of output devices.

[0194] Mass storage device 1007 is connected to central processing unit 1001 via a mass storage controller (not shown) connected to system bus 1005. Mass storage device 1007 and its associated computer-readable media provide non-volatile storage for computer device 1000. That is, mass storage device 1007 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drive.

[0195] Computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile optical disc (DVD), or solid-state drives (SSD), other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1004 and the mass storage device 1007 mentioned above can be collectively referred to as memory.

[0196] According to various embodiments of this application, the computer device 1000 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1000 can be connected to the network 1012 via the network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 can be used to connect to other types of networks or remote computer systems (not shown).

[0197] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU to implement the hypernetic network training method described above.

[0198] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the hypernetic network training method described in the above embodiments.

[0199] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0200] It should be noted that the hypernetic network training device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the hypernetic network training method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the hypernetic network training device and the hypernetic network training method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0201] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0202] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0203] The above description is merely an exemplary embodiment that can be implemented in this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training hypernetworks, characterized in that, The method includes: The first sample set is used to train the supernetwork in the first stage. The supernetwork includes M subnetworks. The subnetworks share network parameters with the supernetwork. M is an integer greater than 1. After the first stage of training is completed, G second sub-networks are sampled from the M sub-networks, and N first sub-networks that meet the first performance requirements are filtered out from the G second sub-networks, where N is an integer greater than 1 and less than M; Cluster analysis is performed on the N first sub-networks to obtain K central sub-networks located at K cluster centers, where K is a positive integer less than or equal to N; The second sample set is used to perform the second stage training on the K central sub-networks, and the network parameters in the supernetwork are adjusted until the error of the supernetwork during the second stage training converges, thus obtaining the trained supernetwork. The first sample set and the second sample set include at least one of the following: image samples, video samples, audio samples, and text samples.

2. The method according to claim 1, characterized in that, The step of extracting N first sub-networks from the M sub-networks includes: Using a uniform sampling method, N first sub-networks that meet the first performance requirements are extracted from the M sub-networks; The uniform sampling method refers to sampling each layer in the neural network structure using the same scaling factor, where the scaling factor indicates the width ratio of the sub-network to the corresponding layers of the super-network.

3. The method according to claim 2, characterized in that, The step of using a uniform sampling method to extract N first sub-networks that meet the first performance requirements from the M sub-networks includes: Using the uniform sampling method, G second sub-networks are extracted from the M sub-networks, where G is greater than or equal to N and less than M; The performance of the second sub-network is tested using the first test sample, and G performance test results are obtained for the G second sub-networks. Based on the G performance test results, N first sub-networks that meet the first performance requirements are determined from the G second sub-networks.

4. The method according to claim 3, characterized in that, The performance test results include the test error of the second sub-network; The process of using the first test sample to perform performance testing on the second sub-network, and obtaining G performance test results corresponding to the G second sub-networks, includes: The error between the output result of the second sub-network and the actual result is tested using the first test sample, and G test errors corresponding to the G second sub-networks are obtained. The step of determining the N first sub-networks that meet the first performance requirements from the G second sub-networks based on the G performance test results includes: According to the G test errors in ascending order, select the first N second sub-networks corresponding to the first N test errors to obtain the N first sub-networks that meet the first performance requirements.

5. The method according to any one of claims 1 to 4, characterized in that, The first stage of training the hypernetwork using the first sample set includes: During the first phase of training, a baseline network is acquired; For each round of training, a third subnetwork is selected from the M subnetworks; The training samples from the first sample set are input into the baseline network to obtain a first output result; and the training samples are input into the third sub-network to obtain a second output result; Calculate the first error between the second output result and the actual result; Calculate the second error between the first output result and the second output result; Based on the first error and the second error, the network parameters in the third sub-network are adjusted.

6. The method according to claim 5, characterized in that, The adjustment of network parameters in the third sub-network based on the first error and the second error includes: Multiply the first error by the first weight to obtain a first product; and multiply the second error by the second weight to obtain a second product; Summing the first product and the second product yields the weighted error; The network parameters in the third sub-network are adjusted according to the weighted error; wherein the sum of the first weight and the second weight is 1.

7. The method according to claim 6, characterized in that, If both the first weight and the second weight are greater than 0, before calculating the second error between the first output result and the second output result, the method further includes: Calculate the third error between the first output result and the actual result; If the difference between the third error and the first error is less than the difference threshold, the value of the first weight is reset to 1, and the value of the second weight is reset to 0.

8. The method according to any one of claims 1 to 4, characterized in that, The process of using a second sample set to perform a second-stage training on the K central sub-networks until the error of the supernetwork during the second-stage training converges, resulting in a trained supernetwork, includes: The second sample set is used to perform the second stage training on the K central sub-networks; If the error of the supernetwork converges, D subnetworks to be tested are extracted from the M subnetworks, where D is an integer greater than 0 and less than M; The performance of the D sub-networks under test is performed using a second test sample. If the performance of all D sub-networks under test meets the second performance requirement, the training of the supernetwork is considered complete.

9. The method according to claim 8, characterized in that, The method further includes: If at least one of the D sub-networks under test fails to meet the second performance requirement, the K central sub-networks are trained again using a third sample set until the performance of all resampled sub-networks under test in the supernetwork meets the second performance requirement.

10. A supercomputer training device, characterized in that, The device includes: The training module is used to perform a first-stage training on the supernetwork using a first sample set. The supernetwork includes M subnetworks, and the subnetworks share network parameters with the supernetwork. M is an integer greater than 1. The sampling module is used to sample G second sub-networks from the M sub-networks after the first stage of training is completed, and filter out N first sub-networks that meet the first performance requirements from the G second sub-networks, where N is an integer greater than 1 and less than M; The clustering module is used to perform clustering analysis on the N first sub-networks to obtain K central sub-networks located at K cluster centers, where K is a positive integer less than or equal to N; The training module is used to perform a second-stage training on the K central sub-networks using a second sample set, and to adjust the network parameters in the supernetwork until the error of the supernetwork during the second-stage training converges, thereby obtaining the trained supernetwork. The first sample set and the second sample set include at least one of the following: image samples, video samples, audio samples, and text samples.

11. A computer device, characterized in that, The computer device includes a processor and a memory connected to the processor. The memory stores program instructions, and when the processor executes the program instructions, it implements the hypernetic network training method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed by a processor, implement the hypernetic network training method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the hypernetic network training method as described in any one of claims 1 to 9.

Images