Visualization method and system for deep neural network loss landscape
By constructing two-dimensional subspaces and adjusting direction vectors, the loss landscape visualization method of the deep neural network model can better capture the characteristics of high-dimensional loss landscapes, improve the credibility of images, and solve the problem of low credibility of lost landscape images in the prior art.
Patent Information
- Application Number
- CN202510176309.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-18
- Publication Date
- 2025-03-18
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art is difficult to fully capture the morphological features and details of the ultra-high-dimensional loss landscape of the deep neural network model, resulting in the credibility of the three-dimensional loss landscape image being questioned.
By constructing a two-dimensional subspace, the first direction vector and the second direction vector are randomly generated, and these direction vectors are adjusted according to the sample number index of the two-dimensional parameter subspace, the model parameters are updated along the iterative direction of the training parameters, and finally the three-dimensional loss landscape image is drawn according to the loss value and the two-dimensional subspace.
It can more comprehensively capture the morphological features and details of the original ultra-high-dimensional loss landscape, improve the credibility of the three-dimensional loss landscape images, and enhance the ability to interpret the generalization performance of deep neural network models.
Smart Images
Figure CN119648926A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of network model analysis, and specifically relates to a method and system for visualizing the loss landscape of a deep neural network. Background Art
[0002] In the existing interpretability analysis of artificial intelligence methods such as deep neural networks, visualization technology is considered to be the most intuitive and effective means. It mainly presents the inner working principle of the deep neural network model in a graphical way. This graphical interpretation result lowers the threshold for understanding the "black box" nature of deep neural networks, and thus has wide recognition in practical applications. Therefore, for highly complex deep neural networks, how to visualize their loss landscape is the key to understanding the actual loss performance of the model.
[0003] The loss landscape of a deep neural network is a visualization of the high-dimensional, implicit dependency between the model's loss error and its parameters. However, given the limitations of human visual perception and the visual confusion of high-dimensional space, the original high-dimensional loss landscape image of the model can only be displayed in three-dimensional space at best. Existing methods are unable to partially capture the morphological features and important details of the original ultra-high-dimensional loss landscape, which raises doubts about the authenticity of the loss landscape visualization image and ultimately undermines the credibility of the interpretation results of the model's generalization performance. Summary of the invention
[0004] The present invention provides a method and system for visualizing a deep neural network loss landscape, which more comprehensively captures the morphological features and details of the original ultra-high dimensional loss landscape and improves the credibility of the three-dimensional loss landscape image.
[0005] In order to achieve the above object, the technical solution adopted by the present invention is:
[0006] A first aspect of the present invention provides a method for visualizing a loss landscape of a deep neural network, comprising:
[0007] Obtaining training parameters of the deep neural network model after training, and generating a training parameter vector according to the training parameters of the deep neural network model;
[0008] Randomly generate a first direction vector and a second direction vector; the dimensions of the first direction vector, the second direction vector and the training parameter vector are equal; set the training parameter as the origin, set the first direction vector and the second direction vector as the iteration direction of the training parameter in the deep neural network model, and construct a two-dimensional subspace;
[0009] Calculate a sample proportion index of the two-dimensional parameter subspace, and adjust the first direction vector and the second direction vector according to the sample proportion index;
[0010] Update the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance; obtain the model training data set and input it into the model instance to obtain the prediction result, and calculate the loss value based on the prediction result and the real label of the model training data set;
[0011] Generate a 3D loss landscape image of a deep neural network model based on the loss values and 2D subspace plotting.
[0012] Furthermore, the sample proportion index of the two-dimensional parameter subspace is calculated, and the expression formula is:
[0013]
[0014] In the formula, It is expressed as the actual number of parameter combinations in the two-dimensional subspace; Indicates the sample proportion index; is the factorial of the number of parameters in the two-dimensional subspace.
[0015] Further, adjusting the first direction vector and the second direction vector according to the sample proportion index specifically includes:
[0016] When the sample proportion index is less than the threshold When , the rotation strategy is used to iteratively adjust the first direction vector and the second direction vector, and the expression formula is:
[0017]
[0018] In the formula, is the number of iterations of the rotation strategy; is the first direction vector outputted in the i-th iteration of the rotation strategy; is the second direction vector outputted at the i-th iteration; is the randomly generated first direction vector; is a randomly generated second direction vector; is the rotation angle of the i-th iteration in the rotation strategy; is a unit vector;
[0019] Calculate the sample fraction index of the two-dimensional parameter subspace and repeat the iterative rotation strategy until the sample fraction index is greater than or equal to the threshold Output the first direction vector and the second direction vector.
[0020] Furthermore, when the number of iterations of the rotation strategy reaches the threshold , and the sample proportion index is less than the threshold When , the first direction vector and the second direction vector corresponding to the maximum sample proportion index are output.
[0021] Further, the training parameter vector of the deep neural network model is updated along the iteration direction of the training parameters to obtain a model instance, specifically including:
[0022]
[0023] In the formula, is the training parameter vector of the deep neural network model; It is the first direction vector of the iterative output in the rotation strategy; The second direction vector is the iterative output of the rotation strategy; is the coordinate in the two-dimensional parameter subspace Candidate parameter vector; x and y are coordinate variables of the two-dimensional parameter subspace;
[0024] Select L sets of coordinate points in the two-dimensional parameter subspace and generate L candidate parameter vectors, denoted as , the candidate parameter vector is standardized, and the 2 norm of the candidate parameter vector is equal to the training parameter vector The 2-norm of ;
[0025] Using the coordinates in the two-dimensional parameter subspace The candidate parameter vector replaces the training parameter vector of the deep neural network model to obtain a model instance.
[0026] Furthermore, a three-dimensional loss landscape image of the deep neural network model is generated based on the loss value and the two-dimensional subspace drawing, specifically including:
[0027] A coordinate axis z perpendicular to the two-dimensional parameter subspace is constructed through the origin to construct a three-dimensional parameter space; the coordinate points of the selected L groups of two-dimensional parameter subspaces are traversed, and the loss value is added to the coordinate axis z to obtain a three-dimensional loss landscape image.
[0028] A second aspect of the present invention provides a system for visualizing a loss landscape of a deep neural network, comprising:
[0029] An acquisition module is used to obtain the training parameters of the deep neural network model after training, and generate a training parameter vector according to the training parameters of the deep neural network model;
[0030] A space construction module, used for randomly generating a first direction vector and a second direction vector; the dimensions of the first direction vector, the second direction vector and the training parameter vector are equal; the training parameter is set as the origin, and the first direction vector and the second direction vector are set as the iteration direction of the training parameter in the deep neural network model to construct a two-dimensional subspace;
[0031] An adjustment module, used for calculating a sample proportion index of a two-dimensional parameter subspace, and adjusting the first direction vector and the second direction vector according to the sample proportion index;
[0032] The loss calculation module is used to update the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance; obtain the model training data set and input it into the model instance to obtain the prediction result, and calculate the loss value according to the prediction result and the real label of the model training data set;
[0033] An image drawing module is used to draw and generate a three-dimensional loss landscape image of a deep neural network model based on the loss value and the two-dimensional subspace.
[0034] Furthermore, the adjustment module adjusts the first direction vector and the second direction vector according to the sample proportion index, specifically including:
[0035] When the sample proportion index is less than the threshold When , the rotation strategy is used to iteratively adjust the first direction vector and the second direction vector, and the expression formula is:
[0036]
[0037] In the formula, is the number of iterations of the rotation strategy; is the first direction vector outputted in the i-th iteration of the rotation strategy; is the second direction vector outputted at the i-th iteration; is the randomly generated first direction vector; is a randomly generated second direction vector; is the rotation angle of the i-th iteration in the rotation strategy; is a unit vector;
[0038] Calculate the sample fraction index of the two-dimensional parameter subspace and repeat the iterative rotation strategy until the sample fraction index is greater than or equal to the threshold Output the first direction vector and the second direction vector;
[0039] When the number of iterations of the rotation strategy reaches the threshold , and the sample proportion index is less than the threshold When , the first direction vector and the second direction vector corresponding to the maximum sample proportion index are output.
[0040] Furthermore, the loss calculation module updates the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance, specifically including:
[0041]
[0042] In the formula, is the training parameter vector of the deep neural network model; It is the first direction vector of the iterative output in the rotation strategy; The second direction vector is the iterative output of the rotation strategy; is the coordinate in the two-dimensional parameter subspace Candidate parameter vector; x and y are coordinate variables of the two-dimensional parameter subspace;
[0043] Select L sets of coordinate points in the two-dimensional parameter subspace and generate L candidate parameter vectors, denoted as , the candidate parameter vector is standardized, and the 2 norm of the candidate parameter vector is equal to the training parameter vector The 2-norm of ;
[0044] Using the coordinates in the two-dimensional parameter subspace The candidate parameter vector replaces the training parameter vector of the deep neural network model to obtain a model instance.
[0045] The third aspect of the present invention provides an electronic device including a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the method for visualizing the deep neural network loss landscape described in the first aspect of the present invention.
[0046] Compared with the prior art, the present invention has the following beneficial effects:
[0047] In the present invention, a first direction vector and a second direction vector are randomly generated; the dimensions of the first direction vector, the second direction vector and the training parameter vector are equal; the training parameters are set as the origin, and the first direction vector and the second direction vector are set as the iteration directions of the training parameters in the deep neural network model to construct a two-dimensional subspace; the sample fraction index of the two-dimensional parameter subspace is calculated, and the first direction vector and the second direction vector are adjusted according to the sample fraction index; by adjusting the direction vector in the two-dimensional subspace, the performance changes of the model in different directions can be explored; by constructing a two-dimensional subspace to analyze the deep neural network model, the morphological characteristics and details of the original ultra-high dimensional loss landscape can be more comprehensively captured, and the credibility of the three-dimensional loss landscape image can be improved.
[0048] In the present invention, the training parameter vector of the deep neural network model is updated along the iteration direction of the training parameters to obtain a model instance; a model training data set is obtained and input into the model instance to obtain a prediction result, and a loss value is calculated according to the prediction result and the real label of the model training data set; a three-dimensional loss landscape image of the deep neural network model is generated according to the loss value and the two-dimensional subspace drawing; by analyzing the terrain of the loss landscape, the decision-making process of the deep neural network model in different directions can be understood, the deep neural network model can be prevented from falling into a local optimization loop, and the interpretability of the deep neural network model can be improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1is a flowchart of a method for visualizing a deep neural network loss landscape provided in Example 1 of the present invention;
[0050] Figure 2 This is a visualization effect diagram of the deep neural network loss landscape provided by Example 1 of the present invention. DETAILED DESCRIPTION
[0051] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0052] Example 1
[0053] like Figure 1 As shown, this implementation provides a method for visualizing the loss landscape of a deep neural network, including:
[0054] Obtain training parameters of the trained deep neural network model, and generate a training parameter vector according to the training parameters of the deep neural network model; in this embodiment, the ResNet-56 deep neural network model is selected, and the number of training parameters n is 25583592;
[0055] Randomly generate a first direction vector and a second direction vector; the dimensions of the first direction vector, the second direction vector and the training parameter vector are equal; set the training parameter as the origin, set the first direction vector and the second direction vector as the iteration direction of the training parameter in the deep neural network model, and construct a two-dimensional subspace;
[0056] Calculate the sample proportion index of the two-dimensional parameter subspace, and the expression formula is:
[0057]
[0058] In the formula, It is expressed as the actual number of parameter combinations in the two-dimensional subspace; Indicates the sample proportion index; is the factorial of the number of parameters in the two-dimensional subspace.
[0059] The first direction vector and the second direction vector are adjusted according to the sample proportion index, specifically including:
[0060] When the sample proportion index is less than the threshold When , the rotation strategy is used to iteratively adjust the first direction vector and the second direction vector, and the expression formula is:
[0061]
[0062] In the formula, is the number of iterations of the rotation strategy; is the first direction vector outputted in the i-th iteration of the rotation strategy; is the second direction vector outputted by the i-th iteration; is the randomly generated first direction vector; is a randomly generated second direction vector; is the rotation angle of the i-th iteration in the rotation strategy; is a unit vector;
[0063] Calculate the sample fraction index of the two-dimensional parameter subspace and repeat the iterative rotation strategy until the sample fraction index is greater than or equal to the threshold Output the first direction vector and the second direction vector;
[0064] When the number of iterations of the rotation strategy reaches the threshold , and the sample proportion index is less than the threshold When , the first direction vector and the second direction vector corresponding to the maximum sample proportion index are output.
[0065] The training parameter vector of the deep neural network model is updated along the iteration direction of the training parameters to obtain a model instance, specifically including:
[0066]
[0067] In the formula, is the training parameter vector of the deep neural network model, that is, the training parameter vector learned by the ResNet-56 deep neural network model after completing training in the CIFAR-10 image classification task; It is the first direction vector of the iterative output in the rotation strategy; The second direction vector is the iterative output of the rotation strategy; is the coordinate in the two-dimensional parameter subspace Candidate parameter vector; x and y are coordinate variables of the two-dimensional parameter subspace;
[0068] Select 200 uniformly from the interval [-1,1] Value and y value, can be constructed Group Coordinate points, according to the coordinate points of the two-dimensional parameter subspace, update the training parameter vector of the deep neural network model and generate L candidate parameter vectors, denoted as , in this embodiment, L=40000; the candidate parameter vector is standardized, and the 2 norm of the candidate parameter vector is equal to the training parameter vector The 2-norm of ;
[0069] Using the coordinates in the two-dimensional parameter subspace The candidate parameter vector replaces the training parameter vector of the deep neural network model to obtain a model instance.
[0070] Obtain the model training data set and input it into the model instance to obtain the prediction result, and calculate the loss value based on the prediction result and the actual label of the model training data set;
[0071] The 3D loss landscape image of the deep neural network model is generated based on the loss value and the 2D subspace drawing, including:
[0072] Construct a coordinate axis z perpendicular to the two-dimensional parameter subspace through the origin to construct a three-dimensional parameter space; traverse the selected L groups of coordinate points in the two-dimensional parameter subspace, and add the loss value to the coordinate axis z to obtain a three-dimensional loss landscape image. Figure 2 As shown, a 3D loss landscape image of the ResNet-56 model is plotted.
[0073] In order to verify the effectiveness of this embodiment, the classic two-dimensional linear candidate parameter generation strategy is selected as a comparison benchmark. Through 100 repeated experiments, the average sample proportion index of the generation strategy proposed by the present invention is 0.8, while the average sample proportion index of the benchmark generation strategy is 0.01. The experimental results show that the visualization graph of the deep neural network loss landscape drawn by the present invention has high credibility.
[0074] Example 2
[0075] This embodiment provides a visualization system for a deep neural network loss landscape, the visualization system is used to execute the visualization method described in Example 1, and the visualization system includes:
[0076] An acquisition module is used to obtain the training parameters of the deep neural network model after training, and generate a training parameter vector according to the training parameters of the deep neural network model;
[0077] A space construction module, used for randomly generating a first direction vector and a second direction vector; the dimensions of the first direction vector, the second direction vector and the training parameter vector are equal; the training parameter is set as the origin, and the first direction vector and the second direction vector are set as the iteration direction of the training parameter in the deep neural network model to construct a two-dimensional subspace;
[0078] An adjustment module, used for calculating a sample proportion index of a two-dimensional parameter subspace, and adjusting the first direction vector and the second direction vector according to the sample proportion index;
[0079] The loss calculation module is used to update the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance; obtain the model training data set and input it into the model instance to obtain the prediction result, and calculate the loss value according to the prediction result and the real label of the model training data set;
[0080] An image drawing module is used to draw and generate a three-dimensional loss landscape image of a deep neural network model based on the loss value and the two-dimensional subspace.
[0081] The adjustment module adjusts the first direction vector and the second direction vector according to the sample proportion index, specifically including:
[0082] When the sample proportion index is less than the threshold When , the rotation strategy is used to iteratively adjust the first direction vector and the second direction vector, and the expression formula is:
[0083]
[0084] In the formula, is the number of iterations of the rotation strategy; is the first direction vector outputted in the i-th iteration of the rotation strategy; is the second direction vector outputted at the i-th iteration; is the randomly generated first direction vector; is a randomly generated second direction vector; is the rotation angle of the i-th iteration in the rotation strategy; is a unit vector;
[0085] Calculate the sample fraction index of the two-dimensional parameter subspace and repeat the iterative rotation strategy until the sample fraction index is greater than or equal to the threshold Output the first direction vector and the second direction vector;
[0086] When the number of iterations of the rotation strategy reaches the threshold , and the sample proportion index is less than the threshold When , the first direction vector and the second direction vector corresponding to the maximum sample proportion index are output.
[0087] The loss calculation module updates the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance, specifically including:
[0088]
[0089] In the formula, is the training parameter vector of the deep neural network model; It is the first direction vector of the iterative output in the rotation strategy; The second direction vector is the iterative output of the rotation strategy; is the coordinate in the two-dimensional parameter subspace Candidate parameter vector; x and y are coordinate variables of the two-dimensional parameter subspace;
[0090] Select L sets of coordinate points in the two-dimensional parameter subspace and generate L candidate parameter vectors, denoted as , the candidate parameter vector is standardized, and the 2 norm of the candidate parameter vector is equal to the training parameter vector The 2-norm of ;
[0091] Using the coordinates in the two-dimensional parameter subspace The candidate parameter vector replaces the training parameter vector of the deep neural network model to obtain a model instance.
[0092] Example 3
[0093] This embodiment provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the method for visualizing the deep neural network loss landscape described in Example 1.
[0094] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0095] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0096] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0097] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0098] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for visualizing the loss landscape of a deep neural network, characterized in that: include: Obtaining training parameters of the deep neural network model after training, and generating a training parameter vector according to the training parameters of the deep neural network model; Randomly generate a first direction vector and a second direction vector; the dimensions of the first direction vector, the second direction vector and the training parameter vector are equal; set the training parameter as the origin, set the first direction vector and the second direction vector as the iteration direction of the training parameter in the deep neural network model, and construct a two-dimensional subspace; Calculate a sample proportion index of the two-dimensional parameter subspace, and adjust the first direction vector and the second direction vector according to the sample proportion index; Update the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance; obtain the model training data set and input it into the model instance to obtain the prediction result, and calculate the loss value based on the prediction result and the real label of the model training data set; Generate a 3D loss landscape image of a deep neural network model based on the loss values and 2D subspace plotting.
2. The method for visualizing the loss landscape of a deep neural network according to claim 1, characterized in that Calculate the sample number index of the two-dimensional parameter subspace, the expression formula is: ； In the formula, It is expressed as the actual number of parameter combinations in the two-dimensional subspace; Indicates the sample proportion index; is the factorial of the number of parameters in the two-dimensional subspace.
3. The method for visualizing the loss landscape of a deep neural network according to claim 1, characterized in that The first direction vector and the second direction vector are adjusted according to the sample proportion index, specifically including: When the sample proportion index is less than the threshold When , the rotation strategy is used to iteratively adjust the first direction vector and the second direction vector, and the expression formula is: ； In the formula, is the number of iterations of the rotation strategy; is the first direction vector outputted in the i-th iteration of the rotation strategy; is the second direction vector outputted by the i-th iteration; is the randomly generated first direction vector; is a randomly generated second direction vector; is the rotation angle of the i-th iteration in the rotation strategy; is a unit vector; Calculate the sample fraction index of the two-dimensional parameter subspace and repeat the iterative rotation strategy until the sample fraction index is greater than or equal to the threshold Output the first direction vector and the second direction vector.
4. The method for visualizing the loss landscape of a deep neural network according to claim 3, characterized in that When the number of iterations of the rotation strategy reaches the threshold , and the sample proportion index is less than the threshold When , the first direction vector and the second direction vector corresponding to the maximum sample proportion index are output.
5. The method for visualizing the loss landscape of a deep neural network according to claim 3 or 4, characterized in that: The training parameter vector of the deep neural network model is updated along the iteration direction of the training parameters to obtain a model instance, specifically including: ； In the formula, is the training parameter vector of the deep neural network model; It is the first direction vector of the iterative output in the rotation strategy; The second direction vector is the iterative output of the rotation strategy; is the coordinate in the two-dimensional parameter subspace Candidate parameter vector; x and y are coordinate variables of the two-dimensional parameter subspace; Select L sets of coordinate points in the two-dimensional parameter subspace and generate L candidate parameter vectors, denoted as , the candidate parameter vector is standardized, and the 2 norm of the candidate parameter vector is equal to the training parameter vector The 2-norm of ; Using the coordinates in the two-dimensional parameter subspace The candidate parameter vector replaces the training parameter vector of the deep neural network model to obtain a model instance.
6. The method for visualizing the loss landscape of a deep neural network according to claim 1, characterized in that The 3D loss landscape image of the deep neural network model is generated based on the loss value and the 2D subspace drawing, including: A coordinate axis z perpendicular to the two-dimensional parameter subspace is constructed through the origin to construct a three-dimensional parameter space; the coordinate points of the selected L groups of two-dimensional parameter subspaces are traversed, and the loss value is added to the coordinate axis z to obtain a three-dimensional loss landscape image.
7. A system for visualizing loss landscapes of deep neural networks, characterized in that include: An acquisition module is used to obtain the training parameters of the deep neural network model after training, and generate a training parameter vector according to the training parameters of the deep neural network model; A space construction module, used for randomly generating a first direction vector and a second direction vector; the dimensions of the first direction vector, the second direction vector and the training parameter vector are equal; the training parameter is set as the origin, and the first direction vector and the second direction vector are set as the iteration direction of the training parameter in the deep neural network model to construct a two-dimensional subspace; An adjustment module, used for calculating a sample proportion index of a two-dimensional parameter subspace, and adjusting the first direction vector and the second direction vector according to the sample proportion index; The loss calculation module is used to update the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance; obtain the model training data set and input it into the model instance to obtain the prediction result, and calculate the loss value according to the prediction result and the real label of the model training data set; An image drawing module is used to draw and generate a three-dimensional loss landscape image of a deep neural network model based on the loss value and the two-dimensional subspace.
8. The system for visualizing the loss landscape of a deep neural network according to claim 7, characterized in that The adjustment module adjusts the first direction vector and the second direction vector according to the sample proportion index, specifically including: When the sample proportion index is less than the threshold When , the rotation strategy is used to iteratively adjust the first direction vector and the second direction vector, and the expression formula is: ； In the formula, is the number of iterations of the rotation strategy; is the first direction vector outputted in the i-th iteration of the rotation strategy; is the second direction vector outputted by the i-th iteration; is the randomly generated first direction vector; is a randomly generated second direction vector; is the rotation angle of the i-th iteration in the rotation strategy; is a unit vector; Calculate the sample fraction index of the two-dimensional parameter subspace and repeat the iterative rotation strategy until the sample fraction index is greater than or equal to the threshold Output the first direction vector and the second direction vector; When the number of iterations of the rotation strategy reaches the threshold , and the sample proportion index is less than the threshold When , the first direction vector and the second direction vector corresponding to the maximum sample proportion index are output.
9. The system for visualizing the loss landscape of a deep neural network according to claim 8, characterized in that The loss calculation module updates the training parameter vector of the deep neural network model along the iteration direction of the training parameters to obtain a model instance, specifically including: ； In the formula, is the training parameter vector of the deep neural network model; It is the first direction vector of the iterative output in the rotation strategy; It is the second direction vector outputted iteratively in the rotation strategy; is the coordinate in the two-dimensional parameter subspace Candidate parameter vector; x and y are coordinate variables of the two-dimensional parameter subspace; Select L sets of coordinate points in the two-dimensional parameter subspace and generate L candidate parameter vectors, denoted as , the candidate parameter vector is standardized, and the 2 norm of the candidate parameter vector is equal to the training parameter vector The 2-norm of ; Using the coordinates in the two-dimensional parameter subspace The candidate parameter vector replaces the training parameter vector of the deep neural network model to obtain a model instance.
10. An electronic device comprising a storage medium and a processor; the storage medium is used to store instructions; characterized in that: The processor is used to operate according to the instructions to execute the method for visualizing the loss landscape of a deep neural network according to any one of claims 1 to 6.