Data visualization system for neural network training process
By employing multi-layer data acquisition, activity quantification, and adaptive downsampling techniques, combined with visual heatmap generation and an interactive interface, the problems of insufficient transparency and low computational efficiency during neural network training are solved, enabling real-time monitoring and anomaly detection, thereby improving training quality and efficiency.
Patent Information
- Application Number
- CN202511513578.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-22
- Publication Date
- 2026-01-20
AI Technical Summary
The existing neural network training process lacks a real-time multi-layer data synchronization acquisition mechanism, which makes it impossible to effectively monitor the state information of each layer. This makes it difficult to detect abnormal states during the training process, such as gradient vanishing and learning stagnation. Furthermore, existing visualization tools are computationally inefficient when dealing with large-scale neural networks.
The system employs hook functions to synchronously collect feature maps, gradients, and weights from each monitoring layer. It then uses an activity metric module to calculate feature changes, weight updates, and gradient strength indices. Combined with adaptive spatial downsampling and visualization heatmap generation, an interactive visualization interface is constructed to achieve real-time monitoring and anomaly detection of multi-layer data.
It achieves full transparency and real-time monitoring of the neural network training process, improves training efficiency and quality, can promptly detect abnormal states during training, and has strong scalability and adaptability.
Smart Images
Figure CN121365702A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer vision, more particularly, it relates to a data visualization system for neural network training process. BACKGROUND
[0002] With the rapid development of deep learning technology, neural network models have achieved remarkable results in computer vision, natural language processing, medical image analysis and other fields. However, the training process of neural network is usually regarded as a black box operation, and researchers have difficulty in intuitively understanding the learning state and feature extraction process of each layer inside the network. The existing training monitoring methods mainly rely on loss function curve, accuracy index and other macro statistical information, and lack the ability of fine-grained visualization analysis of the activation state of the internal layers of the network.
[0003] Traditional neural network training process monitoring technology has many limitations. The existing method cannot display the activation mode changes of each layer feature map in real time, which makes researchers unable to timely discover abnormal states in the training process, including gradient vanishing, overfitting and learning stagnation; it lacks effective multi-layer data synchronous acquisition mechanism, and it is difficult to obtain complete state information of each layer in the training process; the existing visualization tools face the problem of low computational efficiency when processing large-scale neural networks, and cannot meet the performance requirements of real-time monitoring.
[0004] In view of the above technical problems, it is urgent to develop a data visualization system that can realize the comprehensive transparency of neural network training process. The system needs to have real-time multi-layer data acquisition capability, efficient data processing and visualization generation capability, and intelligent abnormal detection and early warning function, so as to provide comprehensive monitoring and analysis support for neural network training process, and improve the training efficiency and quality of deep learning model. SUMMARY
[0005] The present application provides a data visualization system for neural network training process, which solves the technical problems of insufficient transparency of training process and real-time visualization performance bottleneck in related technologies.
[0006] The present application provides a data visualization system for neural network training process, which includes: A multi-layer data acquisition module, which synchronously acquires the feature map, gradient and weight of each monitoring layer during neural network training by using hook function, and outputs an original data matrix set; An activity quantification module, which calculates the feature change, weight update and gradient intensity index according to the original data matrix set, and generates a layer activity score vector by weighted fusion; An adaptive spatial downsampling module, which formulates an adaptive sampling strategy according to the layer activity score vector, keeps the original feature map of high activity layer, and performs secondary downsampling on low activity layer, and outputs the processed feature map; a visualization heat map generation module, which performs channel fusion and normalization on the processed feature map, converts the processed feature map into a heat map and calculates statistical features, and outputs a heat map data structure; a visualization data packet construction module, which calculates rendering priorities based on the heat map data structure and the active degree score vectors of the layers, generates color mapping and annotations, and constructs a visualization data packet containing a heat map, color information, priorities, and metadata; an interactive interface rendering module, which renders tasks in priority order, renders images and generates annotations in combination with the content of the visualization data packet, and outputs a visualization interface of the training process.
[0007] In a preferred embodiment, the calculation method of the feature change intensity indicator in the activity quantification module is as follows: feature map data of each monitoring layer in the original data matrix set is obtained; the variance calculation method is used to evaluate the activation change degree in each feature channel of each feature map data, and the variance values of the channels are arithmetically averaged to obtain the feature change intensity indicator of the current monitoring layer.
[0008] In a preferred embodiment, the calculation method of the weight update intensity indicator in the activity quantification module is as follows: weight change data of each monitoring layer is extracted from the original data matrix set; a weight change matrix is constructed based on the weight change data, the square values of each element in the weight change matrix are accumulated and summed, and the square root of the accumulated result is calculated to obtain the weight update amplitude indicator of the monitoring layer.
[0009] In a preferred embodiment, the determination method of the adaptive sampling strategy in the adaptive spatial downsampling module is as follows: the original spatial resolution is maintained for high active layers with an activity score greater than a second activity threshold value to retain important detail information, 2 times downsampling processing is used for moderately active layers with an activity score between the second activity threshold value and a first activity threshold value, and 4 times downsampling processing is used for low active layers with an activity score less than the first activity threshold value.
[0010] In a preferred embodiment, the processing method of channel fusion in the visualization heat map generation module is as follows: the sampled feature map is obtained, channel fusion processing is performed on the multi-channel feature map of each monitoring layer at each spatial position, the arithmetic mean of all channel activation values at the spatial position is calculated, the activation values of the channels are added and then divided by the total number of channels to obtain the fusion activation intensity of the spatial position, and the visualization heat map data structure is generated after normalization processing.
[0011] In a preferred embodiment, the calculation method of rendering priority in the visualization data packet construction module is: Obtain the training process visualization heat map data structure and the activity score vector; divide the activity score of each monitoring layer by the sum of the activity scores of all monitoring layers to obtain the rendering priority weight of the layer by normalization calculation, and ensure that the sum of the rendering priorities of all layers is 1.
[0012] In a preferred embodiment, the generation method of the color mapping index in the visualization data packet construction module is: Map the fusion activation intensity of the spatial position to the corresponding color index; and map the low activation area, the medium activation area and the high activation area to different color areas according to the activation intensity classification to generate complete color mapping information.
[0013] In a preferred embodiment, a data visualization system for a neural network training process further comprises: An anomaly detection module performs anomaly detection based on time sequence evolution characteristics; calculates the cosine similarity index and the activity change trend parameter of the activation mode of each layer between adjacent time points according to the time sequence data of the activity score vector of each monitoring layer, establishes an anomaly detection rule set including similarity anomaly, activity trend anomaly, change speed anomaly and stability anomaly, performs weighted summation operation on each type of anomaly index through a preset weight, and calculates a comprehensive anomaly score for training process anomaly state recognition.
[0014] In a preferred embodiment, the judgment standard of the anomaly detection module is: Based on the cosine similarity index and the activity change trend parameter of the activation mode of each layer between adjacent time points, set an anomaly detection threshold for state judgment, mark as similarity anomaly when the cosine similarity value of the activation mode between adjacent time points is less than a first similarity threshold, mark as trend anomaly when the absolute value of the activity change slope is greater than a second similarity threshold, mark as speed anomaly when the activity change speed is greater than a first change speed threshold, and mark as stability anomaly when the network overall stability index is less than a second change speed threshold.
[0015] In a preferred embodiment, a computer readable storage medium is used to store computer readable instructions, which can run the above-mentioned data visualization system for a neural network training process when the computer readable instructions are read by a computer.
[0016] The beneficial effects of the present application are: By constructing a complete multi-layer data synchronization acquisition and activity quantification evaluation system, the comprehensive transparency and real-time monitoring of the neural network training process are realized. The system realizes the synchronous acquisition of multi-layer feature data by using the hook function mechanism, and combines the multi-dimensional fusion evaluation method of the feature change intensity index, the weight update amplitude index and the gradient intensity index, which can accurately quantify the training activity state of each layer. Through adaptive spatial downsampling processing and hierarchical priority data packet construction, the system can improve the data processing efficiency while ensuring the visualization quality, and reduce the consumption of computing resources;
[0017] It has strong expansibility and adaptability, and can integrate professional analysis functions according to the specific needs of different application fields. The system supports abnormal detection enhancement function based on time evolution characteristics, which can timely detect abnormal states such as gradient disappearance and learning stagnation in the training process through comprehensive analysis of cosine similarity index and activity change trend parameters, and greatly improves the accuracy of abnormal detection. BRIEF DESCRIPTION OF DRAWINGS
[0018] Figure 1 is a module diagram of a data visualization system for neural network training process of the present application; Figure 2 is an effect diagram in medical image analysis application of the present application. DETAILED DESCRIPTION
[0019] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that the discussion of these implementations is merely meant to provide a better understanding of the subject matter described herein and can include changes, modifications, or additions of elements to the functions and arrangements of elements discussed without departing from the scope of the present disclosure. Various examples can omit, substitute, or add various procedures or components as appropriate, or modify aspects of the described examples. Additionally, some implementations described with respect to a particular example can be incorporated in other examples as well.
[0020] In at least one embodiment of the present application, a data visualization system for neural network training process is disclosed, as shown in Figure 1 includes: A multi-layer data acquisition module synchronously acquires feature maps, gradients and weights of each monitoring layer during neural network training by using a hook function, and outputs an original data matrix set; Specifically, the calculation data of forward propagation and back propagation in the neural network training process is obtained, including feature maps, gradient information and weight parameters of each layer; The system registers data collection hook functions at the 5th, 15th, 25th, 35th, and 45th layers of the neural network (depending on the specific number of layers of the neural network), which are automatically triggered during each forward and backward propagation process. When the network processes a batch of data containing 16 images, the hook functions capture the output feature map data of the specified monitoring layers in real time. Taking the 15th layer as an example, the collector captures a feature map tensor with a size of 16 by 128 by 64 by 64, where 16 represents the batch size, 128 represents the number of feature channels, and 64 by 64 represents the spatial resolution of each feature channel.
[0021] During the backward propagation phase, the hook functions synchronously capture the gradient tensors of the monitoring layers with respect to the loss function, and the gradient data has the same dimension structure as the corresponding feature maps; at the same time, the system calculates the weight change quantity by comparing the weight parameters of the current training iteration with those of the previous iteration, i.e., the weight parameters of the current iteration minus the weight parameters of the previous iteration, to quantify the learning activity level of the network at that layer; The output includes a raw data matrix set containing complete data of 5 monitoring layers, which includes the feature map of each monitoring layer at the current time, the corresponding gradient information, and the weight change quantity; where the feature map represents the output feature of the Lth monitoring layer at time t, the gradient information represents the corresponding gradient data of the Lth monitoring layer at time t, and the weight change quantity represents the change of the weight of the layer; each data item is labeled with the corresponding timestamp and layer identifier.
[0022] The activity quantification module calculates the feature change, weight update, and gradient intensity indicators based on the raw data matrix set and generates a layer activity score vector by weighted fusion; The training activity of each layer is quantitatively evaluated by a multi-dimensional index fusion calculation method; Calculate the feature change intensity indicator: for the feature map of the Lth layer, calculate the variance of the pixel values in each feature channel, and then average the variance values of all channels. Specifically, for the 15th layer with 128 channels and a 64x64 pixel feature map in each channel, the variance of 4096 pixel values in each channel is calculated by traversing all 128 channels, then the 128 variance values are added and divided by 128 to obtain the average variance, which is the feature change intensity indicator of the layer. Each channel of the 64x64 feature map represents the response intensity distribution of the layer on the input data in that feature dimension.
[0023] The weight update amplitude index is calculated: the intensity of weight update is quantified by calculating the Frobenius norm of the weight change amount; the specific method is to accumulate the square value of each element in the weight change matrix, and then take the square root of the accumulated result to obtain the overall amplitude of weight update; this index reflects the adjustment degree of the network in the current training iteration, and the larger the value, the more active the network learning.
[0024] The gradient intensity index is calculated: the intensity of the gradient is evaluated by calculating the Frobenius norm of the gradient tensor; the specific method is to accumulate the square value of each element in the three-dimensional gradient tensor, and then take the square root of the accumulated result to obtain the overall intensity of the gradient; this index reflects the sensitivity of the loss function to the parameters of the layer, and the larger the value, the more obvious the influence of the layer on the final output.
[0025] The three indexes are weighted and fused according to the preset weights to obtain the activity score of each layer; the specific method is to multiply the feature change intensity index, the weight update amplitude index and the gradient intensity index by the weight coefficient respectively, then add the three weighted results and process them through the softmax function to obtain the final activity score; this weighted fusion method ensures that the feature change dominates in the activity evaluation, while taking into account the contribution of weight update and gradient intensity; The activity score vector of the 5 monitoring layers is output, including the training activity score value of each monitoring layer; for example, the activity score vector is [0.65, 0.78, 0.82, 0.71, 0.59], which respectively corresponds to the training activity scores of the 5th layer, the 15th layer, the 25th layer, the 35th layer and the 45th layer.
[0026] The adaptive spatial downsampling module formulates an adaptive sampling strategy according to the activity score vector of each layer, keeps the high activity score layers unchanged, and performs secondary downsampling on the low activity score layers to output the processed feature maps. Specifically, the downsampling strategy is determined according to the activity score of each layer; the specific method is to classify the activity scores according to the preset threshold interval: the layers with an activity score greater than 0.8 adopt the strategy of keeping the original resolution, i.e. no downsampling processing is performed, and the complete spatial information is preserved; the layers with an activity score between 0.5 and 0.8 adopt 2 times downsampling, i.e. the spatial resolution of the original feature map is reduced to half of the original; the layers with an activity score less than 0.5 adopt 4 times downsampling, i.e. the spatial resolution is reduced to one fourth of the original.
[0027] For each layer of feature map, the corresponding downsampling method is adopted according to its activity score; for the layers that need 2 times downsampling, the average pooling method is adopted to reduce the spatial resolution of the original feature map to half of the original; for the layers that need 4 times downsampling, the spatial resolution is reduced to one fourth of the original. The specific process is to divide the original feature map into several non-overlapping square regions, and the side length of each region is equal to the downsampling multiple; then the average value of all pixel values in each region is calculated, and the average value is taken as the pixel value of the corresponding position after downsampling; this average pooling method can retain the main information of the original feature map while reducing the data volume; each position coordinate of the feature map after downsampling corresponds to the average activation intensity of a pooling window region in the original feature map.
[0028] The output sampled feature map set contains 5 monitoring layer feature map data after adaptive downsampling processing; among them, the layer with high activity keeps relatively high spatial resolution, and the layer with low activity is downsampled to a greater extent; the total data volume is greatly reduced compared with the original data, and the calculation efficiency is improved.
[0029] The visual heat map generation module performs channel fusion and normalization on the processed feature map, converts it into a heat map and calculates statistical features, and outputs a heat map data structure; Specifically, for a three-dimensional feature map with multiple channels in the Lth layer, the arithmetic mean of the activation values of all channels is calculated at each spatial position; the specific method is to add the activation values of all channels at each spatial position of the feature map, and then divide by the total number of channels to obtain the fusion activation intensity at that position; this channel fusion method can compress multi-dimensional feature information into two-dimensional spatial activation distribution, which is convenient for subsequent visualization processing.
[0030] The activation intensity distribution of each layer is normalized, and the maximum and minimum value normalization method is used to map the numerical range to the interval of 0 to 1 to obtain the normalized activation intensity. This normalization processing ensures that the activation intensities of different levels have the same numerical range, which is convenient for unified visualization display and comparative analysis; at the same time, the statistical features of each layer heat map are calculated, including the mean and standard deviation; these statistical features provide quantitative reference indicators for subsequent data analysis and anomaly detection.
[0031] The output training process visual heat map data structure contains the normalized spatial activation intensity distribution and its statistical features of the 5 levels. The data structure contains the normalized heat map, mean and standard deviation information of each level; each heat map clearly reflects the image region that the corresponding layer focuses on under the current training state, providing an intuitive visual representation for the training process;
[0032] The visualization data packet construction module calculates the rendering priority, generates color mapping and annotations, and constructs a visualization data packet containing heat maps, color information, priority and metadata based on the heat map data structure and the activity score vector of each layer; Specifically, the rendering priority is calculated based on the activity score of each layer, which is obtained by dividing the activity score of each layer by the sum of the activity scores of all five monitoring layers. A color mapping index is generated for each layer of the heat map. The normalized activation intensity value (ranging from 0 to 1) is mapped to a 256-level color index. The specific method is to multiply the activation intensity value by 255 and then take the integer part to obtain an integer index between 0 and 255. The color mapping strategy is as follows: low activation area (index 0-85) is mapped to blue, indicating that the neuron activation in this area is low; medium activation area (index 86-170) is mapped to green, indicating moderate neuron activation; high activation area (index 171-255) is mapped to red, indicating high neuron activation. A training state annotation is generated for each layer, including the activity score and the learning state description. The annotation content includes the layer identifier, the activity score value of the current training iteration, and the learning state description based on the activity level, such as high activity, medium activity, or low activity.
[0033] At the same time, a JSON format metadata structure containing activity indicators, statistical features, and other information is constructed. The metadata includes the timestamp of the current training iteration, the activity score array of all monitoring layers, and the statistical features (mean and standard deviation) of each layer heat map. This structured metadata provides a complete information base for subsequent data analysis, historical review, and performance monitoring.
[0034] A plurality of resolution versions are generated for each heat map through bilinear interpolation to adapt to different display requirements. Specifically, a low-resolution version of 32x32 pixels is generated for overview display, a medium-resolution version of 64x64 pixels is generated for regular display, and a high-resolution version of 128x128 pixels is generated for detailed analysis. The bilinear interpolation method ensures smooth transition and information fidelity between different resolution versions.
[0035] The output training state visualization data package contains complete color mapping information, rendering priority, state annotation, metadata, and multi-resolution heat maps.
[0036] An interactive interface rendering module sorts the rendering tasks according to priority, renders images based on the visualization data package content, and generates annotations to output a visual interface of the training process. Specifically, the rendering tasks are sorted according to the rendering priority of each layer. The specific method is to arrange the rendering priorities of the five monitoring layers in descending order according to the numerical value to obtain the rendering order from high priority to low priority. The visualization content of high priority layers is rendered first to ensure timely display of important information. When the user inputs a specific interaction instruction, the system executes the corresponding response logic: if the user inputs an instruction to view a specific training iteration, the system retrieves the visualization data package of the corresponding time from the historical data cache, including the heat map, activity score, and statistical information of that time; if the user inputs an instruction to focus on a specific layer, the system calls the color mapping index and metadata of that layer for highlighted display, highlighting the activation pattern and training state of that layer; if the user inputs an instruction to filter by activity, the system only displays the layers with an activity score greater than the user-set threshold, filtering out the layers with low activity to highlight the key points.
[0037] The interface layout uses a grid method to simultaneously render heat maps of multiple layers, and the display size of each heat map is dynamically adjusted according to its priority; the specific adjustment method is to take the basic display size as the benchmark and proportionally enlarge the display size according to the rendering priority of the layer; the higher the priority of the layer, the larger the display area it obtains, ensuring that important information is more prominently displayed; the basic display size is the minimum display size for all layers, and the priority magnification factor controls the size adjustment range; The corresponding training state annotations and real-time activity indicators are displayed below each heat map; the system integrates interactive controls such as timeline sliders, layer selectors, and activity filters, supporting users to dynamically observe the evolution of feature learning for each layer during training; The output neural network training process visualization interface includes a priority-ordered sequence of layer rendering commands, corresponding training state annotation information, dynamic layout information, interactive response logic, and real-time update mechanisms. This interface includes annotation information and display size information for each monitored layer, providing users with an intuitive, real-time, and interactive neural network training process visualization experience.
[0038] In an embodiment of the present application, abnormal situations that may occur during training, such as gradient disappearance, learning stagnation, and overfitting, need to be discovered and warned in a timely manner by analyzing the time evolution characteristics of the training process; by introducing time evolution feature analysis and multi-dimensional anomaly pattern recognition technology, an enhanced function module for training process anomaly detection is constructed; Specifically, historical time series data is obtained, including feature map data, gradient information, weight change, activity score, and other key indicators in past training iterations; based on the visualization heat map data structure, historical time series data is input to realize intelligent identification and early warning of abnormal states of the training process, thereby improving the intelligent level of training process monitoring; The specific implementation is as follows: Training evolution feature extraction based on historical time series data; analyze the evolution pattern of the training process through multi-dimensional time series feature extraction method; calculate the cosine similarity of the activation pattern of each layer between adjacent time points, the specific method is to carry out dot product operation on the activation vectors of the current time and the previous time, then divide by the product of the module length of the two vectors, get the similarity value between the two activation patterns. The value range is between -1 and 1, the closer to 1, the more similar the activation pattern, the closer to -1, the more opposite the activation pattern; for the Lth layer, the heat map of the current time t and the previous time t-1 is flattened into a vector form, that is, the two-dimensional heat map matrix is rearranged into a one-dimensional vector according to the row or column order, the dimension of the vector is equal to the height multiplied by the width of the heat map; calculate the cosine similarity of the two vectors, that is, the dot product of the two vectors divided by the product of the two vectors.
[0039] Calculate the change trend of the activity of each layer, fit the activity change slope in the recent several time windows by linear regression; The specific method is to collect the activity data in the recent N time windows, linear fitting with time as independent variable and activity as dependent variable, the calculated slope reflects the change trend of the activity; Positive slope indicates that the activity shows an upward trend, negative slope indicates that the activity shows a downward trend, and the absolute value of the slope reflects the degree of change; Select the activity data of the last N training iterations (N=50) to construct a time series, and fit the linear trend by least squares method; The specific calculation method is to use the standard formula of least squares method, taking the index of time series as independent variable and the activity value as dependent variable, and calculating the slope coefficient of linear regression; Calculate the change speed of the activation pattern of each layer, measure the difference between the activation vectors of adjacent time points by Euclidean distance; The specific method is to calculate the difference vector of the activation vectors of the current time and the previous time, and then calculate the two norm of the difference vector to get the change amplitude of the activation pattern; The larger the value, the more intense the change of the activation pattern, the smaller the value, the more gentle the change of the activation pattern; Calculate the change speed of the activity between adjacent training iterations, that is, the absolute value of the difference between the current time activity and the previous time activity.
[0040] Evaluate the stability of the training state of each layer by calculating the activation variance of the last S time points. The specific method is to collect the activation vectors of the last S time points, calculate the average of these vectors as a reference, then calculate the square of the Euclidean distance between each time point's activation vector and the average, and finally average all distance squares to get the stability index. The smaller the value, the more stable the training state, the larger the value, the greater the fluctuation of the training state. Calculate the overall stability index of the network by weighted average of the activation pattern similarity of all monitoring layers, that is, multiply the activity score of each layer by the similarity index of that layer, then sum all layers, and finally normalize by dividing the sum of all layer activity scores;
[0041] where the activity score of each layer is used as a weight, ensuring that layers with high activity contribute more to the overall stability.
[0042] The output temporal evolution features include the activation pattern similarity of each layer, the activity change trend, the overall stability of the network, and the learning speed, providing a comprehensive temporal feature basis for anomaly detection.
[0043] Based on multi-dimensional anomaly pattern recognition, the training state anomaly detection is performed by processing the evolution feature data using a multi-dimensional anomaly pattern recognition method. First, a set of anomaly detection rules is established, including the following four types of anomaly patterns: Similarity anomaly: When the activation pattern similarity between adjacent time points is too low, it indicates that the network learning has a sudden change. The specific judgment method is to compare the calculated similarity value with the pre-set similarity anomaly threshold. If the similarity value is less than the threshold 0.7, it is marked as a similarity anomaly, otherwise it is marked as a normal state.
[0044] Activity trend anomaly: When the activity change slope is too large, it indicates that the training process is unstable. The specific judgment method is to compare the absolute value of the activity change slope with the pre-set trend anomaly threshold. If the absolute value of the slope is greater than the threshold 0.05, it is marked as a trend anomaly, otherwise it is marked as a normal state.
[0045] Change speed anomaly: When the activity change speed is too fast, it indicates that the network is in shock. The specific judgment method is to compare the activity change speed with the pre-set speed anomaly threshold. If the change speed is greater than the threshold 0.1, it is marked as a speed anomaly, otherwise it is marked as a normal state.
[0046] Stability anomaly: When the overall stability of the network is too low, it indicates that the training is divergent. The specific judgment method is to compare the overall stability index of the network with the pre-set stability anomaly threshold. If the stability index is less than the threshold 0.8, it is marked as a stability anomaly, otherwise it is marked as a normal state.
[0047] The comprehensive anomaly score is calculated by weighted summation of all types of anomaly indicators. The specific method is to multiply the four types of anomaly marks by the corresponding weight coefficients, and then sum them up to get the comprehensive anomaly score. The weight settings are similarity anomaly weight 0.3, trend anomaly weight 0.2, speed anomaly weight 0.2, and stability anomaly weight 0.3, highlighting the importance of similarity and stability. The numerical range of the comprehensive anomaly score is between 0 and 1, and the higher the value, the more serious the anomaly.
[0048] Calculate the anomaly score of each layer; for the Lth layer, consider factors such as stability deviation, activity anomaly degree, and gradient anomaly; the specific calculation method is to multiply the absolute value of the difference between the similarity index of the layer and the historical similarity average by the weight, add the absolute value of the difference between the activity of the layer and the historical activity average multiplied by the weight, and add the absolute value of the difference between the change speed of the layer and the historical change speed average multiplied by the weight; the average of the historical data is a benchmark value obtained by calculating the average of the corresponding index in the past period, which is used to measure the deviation of the current state from the normal state.
[0049] Calculate the overall network anomaly score by weighted average of the activity of all layers; the specific method is to multiply the activity score of each layer by the anomaly score of the layer, then sum all 5 monitoring layers, and finally normalize by dividing the sum of all layer activity scores to more accurately reflect the overall abnormal state of the network; Output the anomaly detection result, including the overall network anomaly score, activity decline anomaly label, learning stagnation anomaly label, and anomaly score of each layer. The result set contains the overall anomaly score, various anomaly labels, and the anomaly score of each monitoring layer, providing comprehensive anomaly detection information for training process monitoring.
[0050] Intelligent early warning visualization interface generation based on anomaly detection results; Anomaly detection result and neural network training process visualization interface; integrate the anomaly detection result into the training monitoring interface through the intelligent early warning visualization method to realize the intuitive display and early warning of the abnormal state; build an abnormal state color coding mapping table to map different types and severity of abnormal states to corresponding color identifiers; the specific method is to set the anomaly score threshold interval: normal state (score 0-0.3) is mapped to green, slight anomaly (score 0.3-0.6) is mapped to yellow, moderate anomaly (score 0.6-0.8) is mapped to orange, and severe anomaly (score 0.8-1.0) is mapped to red. For specific anomaly types, similarity anomaly uses a blue border identifier, trend anomaly uses a purple border identifier, speed anomaly uses a cyan border identifier, and stability anomaly uses a brown border identifier.
[0051] Generate an abnormal state time evolution curve to show the trend of anomaly scores of each layer with training iterations; the specific method is to collect the anomaly score data of the last 100 training iterations to construct a time series data matrix, where the rows represent training iterations and the columns represent monitoring layers; show the evolution trajectory of each layer's anomaly score in the form of a line chart, use different colored lines to distinguish different levels, and use line thickness to reflect the activity weight of the layer. When the anomaly score exceeds the preset threshold, mark the anomaly at the corresponding time point and display the specific anomaly type and score value;
[0052] An abnormality early warning information panel is constructed to display the current detected abnormal state and early warning information in real time; the specific method is to generate a structured early warning information list according to the results of abnormality detection; each early warning information includes abnormality occurrence time, abnormality type, involved level, abnormality score, severity level and suggested treatment measures; the early warning information is sorted according to severity and occurrence time, and severe abnormalities are displayed preferentially, and abnormalities of the same severity are arranged in reverse chronological order; the early warning panel supports filtering and searching functions of abnormality information, and the user can filter and display the content according to the abnormality type, level, time range and the like; An abnormality state indicator is integrated into the original heat map visualization interface, and an abnormality state indicator is added to the border and title area of each level heat map; the specific method is to display the corresponding color coding in the heat map border and add the abnormality state icon and score value in the title area according to the abnormality score and abnormality type of the layer; when the mouse hovers over the abnormality indicator, a detailed abnormality information prompt box is displayed, including abnormality type, score, detection time and historical trend information; at the same time, a global abnormality state indicator bar is added at the top of the interface to display the overall abnormality score and state of the network, and a sound or flashing reminder is triggered when the overall abnormality score exceeds the threshold; An intelligent early warning visualization interface integrating abnormality detection function is output, including abnormality state color coding, time sequence evolution curve, early warning information panel and abnormality state indicator; the interface includes complete abnormality detection visualization components, early warning reminder mechanism and interactive functions, and provides comprehensive abnormality monitoring and intelligent early warning capability for the neural network training process, realizing a complete technical closed loop from abnormality detection to visualization display.
[0053] In an embodiment of the present application, a specific lung CT image tumor detection application example is provided; the example is based on a ResNet-50 network architecture and is used for detecting tumor regions in a 512x512 pixel lung CT image; the training data set includes 10,000 labeled lung CT images, including 5,000 normal images and 5,000 images containing tumors; In this application scenario, the visualization system monitors the training process of the network at the 10th layer, the 20th layer, the 30th layer, the 40th layer and the 50th layer, and analyzes whether the network has learned medically meaningful features in real time; the medical prior knowledge is defined in advance: the central region of the lung is a high attention region (weight 1.0), the edge region of the lung is a medium attention region (weight 0.8), and the peripheral region of the lung is a low attention region (weight 0.5); The medical attention degree data of each layer in the training process is shown in Table 1: Table 1: Medical attention degree analysis results of each monitoring layer
[0054] The network in Table 1 has a medical important region attention value in the range of 0-1, and the higher the value, the more the network pays attention to the medical related region; similarly, the lower the attention in the non-medical region, the less the network pays attention to the non-medical related region; the attention bias index: ; range-1 to 1, positive value indicates bias towards medical region; spatial distribution entropy: the dispersion degree of attention distribution, the lower the value, the more concentrated the attention; activity score: the overall training activity of the layer, the value range is 0-1; The medical learning effect evaluation index data is shown in Table 2: Table 2 Medical learning effect evaluation results at different iteration stages in the training process
[0055] The medical attention trend in Table 2 represents the linear fitting slope of medical attention in the last 30 iterations, the value range is 0-1, the higher the value, the better the learning trend; the bias stability index represents the stability of the attention bias; the medical learning efficiency represents the medical attention growth per iteration; the inter-layer consistency index represents the consistency of the attention bias of each layer; the comprehensive learning quality score represents the comprehensive evaluation score; the learning state rating represents the grade division based on the comprehensive score (excellent: 90-100 points, good: 70-89 points, general: 50-69 points, poor: <50 points); As shown in Figure 2 , the continuous improvement trend of each medical learning index in the training process is shown. The comprehensive learning quality score (red thick line) is improved from 42.5 points at the beginning to 94.2 points, indicating that the network gradually learns the medical meaningful features. Each sub-index shows a stable upward trend, which verifies the effectiveness and practicality of the application in medical image analysis application.
[0056] The above describes embodiments of the application, but the embodiments are not limited to the specific implementation described above, which is only illustrative and not limiting, and those skilled in the art can make more forms of equivalent embodiments under the inspiration of the embodiments, which are all within the protection scope of the embodiments.
Claims
1. A data visualization system for the neural network training process, characterized in that, include: The multi-layer data acquisition module uses hook functions to synchronously acquire feature maps, gradients, and weights of each monitoring layer during neural network training, and outputs a set of raw data matrices. The activity measurement module calculates feature changes, weight updates, and gradient strength indices based on the original data matrix set, and performs weighted fusion to generate activity score vectors for each layer. The adaptive spatial downsampling module formulates an adaptive sampling strategy based on the activity score vector of each layer. It keeps the high-activity layer unchanged and performs secondary downsampling on the low-activity layer, outputting the processed feature map. The visualization heatmap generation module performs channel fusion and normalization on the processed feature map, converts it into a heatmap, calculates statistical features, and outputs the heatmap data structure. The visualization data package construction module, based on the heatmap data structure and the activity score vector of each layer, calculates the rendering priority, generates color maps and annotations, and constructs a visualization data package containing heatmaps, color information, priorities and metadata. The interactive interface rendering module prioritizes rendering tasks, renders images based on the content of the visualization data package, generates annotations, and outputs a visualization interface of the training process.
2. The data visualization system for neural network training process according to claim 1, characterized in that, The calculation method for the feature change intensity index in the active measurement module is as follows: Obtain the feature map data of each monitoring layer in the original data matrix set; For each feature map data, the variance calculation method is used to evaluate the degree of activation change within each feature channel. The variance values of each channel are then arithmetically averaged to obtain the feature change intensity index of the current monitoring layer.
3. The data visualization system for neural network training process according to claim 1, characterized in that, The calculation method for the weight update intensity index in the active metrics module is as follows: Extract the weight change data of each monitoring layer from the original data matrix set; A weight change matrix is constructed based on the weight change data. The square values of each element in the weight change matrix are summed, and the square root of the summation result is taken to obtain the weight update magnitude index of the monitoring layer.
4. A data visualization system for neural network training process according to claim 1, characterized in that, The method for determining the adaptive sampling strategy in the adaptive spatial downsampling module is as follows: For highly active levels with activity scores greater than the second activity threshold, the original spatial resolution is maintained to preserve important details. For moderately active levels with activity scores between the second and first activity thresholds, a 2x downsampling process is applied. For low-activity levels with activity scores less than the first activity threshold, a 4x downsampling process is applied.
5. A data visualization system for neural network training process according to claim 1, characterized in that, The channel fusion processing method in the visualization heatmap generation module is as follows: After obtaining the sampled feature map, channel fusion processing is performed on the multi-channel feature map of each monitoring layer at each spatial location. The arithmetic mean of the activation values of all channels at the spatial location is calculated. The activation values of each channel are added together and divided by the total number of channels to obtain the fusion activation intensity at that spatial location. After normalization processing, a visual heat map data structure is generated.
6. A data visualization system for neural network training process according to claim 1, characterized in that, The method for calculating rendering priority in the visualization data package construction module is as follows: Obtain the data structure of the training process visualization heatmap and the activity score vector; divide the activity score of each monitoring layer by the sum of the activity scores of all monitoring layers, and calculate the rendering priority weight of that layer through normalization to ensure that the sum of the rendering priorities of all layers is 1.
7. A data visualization system for neural network training process according to claim 1, characterized in that, The method for generating the color mapping index in the visualization data package construction module is as follows: The spatial location activation intensity is mapped to the corresponding color index; according to the activation intensity level, low activation region, medium activation region and high activation region are mapped to different colors to distinguish them, generating complete color mapping information.
8. A data visualization system for neural network training process according to claim 1, characterized in that, Also includes: The anomaly detection module performs anomaly detection based on temporal evolution characteristics; Based on the time-series data of the activity score vectors of each monitoring layer, the cosine similarity index and activity change trend parameters of the activation modes of each layer between adjacent time points are calculated. An anomaly detection rule set including similarity anomalies, activity trend anomalies, change rate anomalies, and stability anomalies is established. The various anomaly indicators are weighted and summed through preset weights to calculate a comprehensive anomaly score for anomaly state identification during the training process.
9. A data visualization system for neural network training process according to claim 8, characterized in that, The judgment criteria of the anomaly detection module are as follows: Based on the cosine similarity index of activation modes of each layer between adjacent time points and the activity change trend parameter, anomaly detection thresholds are set for state judgment. When the cosine similarity value of activation modes between adjacent time points is less than the first similarity threshold, it is marked as a similarity anomaly. When the absolute value of the slope of activity change is greater than the second similarity threshold, it is marked as a trend anomaly. When the rate of activity change is greater than the first rate of change threshold, it is marked as a rate anomaly. When the overall network stability index is less than the second rate of change threshold, it is marked as a stability anomaly.
10. A computer-readable storage medium, characterized in that, It is used to store computer-readable instructions that, when read by a computer, enable the execution of a data visualization system for neural network training as described in any one of claims 1-9.