A deep learning-based training model optimization method

Abstract

The application discloses a kind of based on deep learning training model optimization method, comprising: reading training data and model parameter, parameter block is divided according to network layer, constructs parameter block state set;Record stable core point transition shell layer and collapse boundary state, construct accessible generalization manifold shell domain;Parameter block and shell domain marking object are constructed, input improved HyperMixer model, generate shell domain attribution result;Based on parameter increment generates transition trajectory, calculates trajectory arc and identifies shell domain separation risk;Local stable tension subspace is constructed, and reconstruction parameter is generated by executing parameter projection and partition update;Verification processing is executed to reconstruction parameter, and when meeting constraint condition, the training model of output optimization is exported.The application realizes the fine identification and stable optimization of parameter deviation in the training process of deep learning model by constructing accessible generalization manifold shell domain and combining parameter transition trajectory arc constraint and topological back volume reconstruction mechanism.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method for optimizing training models based on deep learning. Background Technology

[0002] With the widespread application of deep learning technology in scenarios such as image recognition, speech processing, natural language processing, industrial prediction, and intelligent control, the optimization quality of the trained model directly affects the model's convergence speed, generalization ability, and deployment stability. Current deep learning model training typically relies on gradient descent, momentum optimization, learning rate decay, weight regularization, batch normalization, early stopping strategies, and validation set performance monitoring to iteratively update and adjust model parameters, gradually reducing training loss and keeping validation performance within an acceptable range.

[0003] However, most existing training optimization methods rely on changes in overall training loss, overall validation loss, accuracy, or global gradient as the basis for adjustment. These methods struggle to identify the spatial migration status of parameter blocks in different network layers across consecutive training rounds, and also fail to determine whether each parameter block remains within a stable parameter region capable of generalization. Especially in the later stages of training, the model may experience a continued decrease in training loss while validation performance begins to fluctuate or decline. At this point, parameter blocks may have gradually migrated from the stable kernel region to the generalization transition shell region, the generalization collapse boundary region, or even the out-of-shell deviation region. Existing methods typically cannot perform fine-grained identification during this latent shift stage.

[0004] When a model experiences a decline in generalization performance, existing technologies typically address this by reducing the learning rate, increasing regularization constraints, stopping training early, or restoring historically optimal model parameters. These methods involve overall regulation and make it difficult to distinguish which network layer parameter blocks still have effective training value and which parameter blocks have deviated from the achievable generalization region. This can easily lead to effective training results being rolled back or invalid parameters continuing to participate in updates.

[0005] Therefore, how to provide a deep learning-based training model optimization method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a deep learning-based training model optimization method. This invention fully utilizes the improved HyperMixer model, the construction of reachable generalization manifold shell domains, parameter transition trajectory radii constraints, and parameter topology rewind reconstruction techniques. It details the optimization process of identifying shell domain affiliation, judging detachment risks, and reconstructing out-of-bounds parameters for network layer parameter blocks during model training. This method avoids relying solely on loss values, learning rate decay, or early stopping strategies for overall control and has the advantages of precise parameter offset identification, high model training stability, strong ability to retain effective training results, and significant improvement in generalization performance.

[0007] A deep learning-based training model optimization method according to an embodiment of the present invention includes:

[0008] Read the training dataset, validation dataset, and deep learning model to be optimized; divide the network layer parameters in the deep learning model to be optimized into parameter blocks; and construct a set of parameter block states.

[0009] During model training, the stable kernel point parameters corresponding to the local optimal validation performance, the generalization transition shell state corresponding to the stable validation loss, and the generalization collapse boundary state corresponding to the increase in validation loss are recorded to construct an reachable generalization manifold shell domain.

[0010] Each parameter block, stable kernel point parameter, generalized transition shell state and generalized collapse boundary state are constructed as corresponding labeled objects and input into the improved HyperMixer model. The shell domain assignment results of each parameter block in the reachable generalized manifold shell domain are generated through dynamic mixing processing.

[0011] In consecutive training rounds, parameter transition trajectories are generated based on parameter increment vectors. The radian of the parameter transition trajectory is generated based on the directional changes between consecutive parameter increment vectors. When the radian of the parameter transition trajectory increases continuously and the corresponding shell domain assignment result is a generalized collapse boundary region or a shell domain deviation region, it is determined that the current model parameters have a risk of detachment from the reachable generalized manifold shell domain.

[0012] In response to the detection of the risk of detachment from the shell domain of the reachable generalized manifold, a locally stable tensor subspace is constructed based on the stable kernel point parameters. The model parameters with the risk of detachment are projected onto the locally stable tensor subspace, and the reconstructed model parameters are generated based on the shell domain assignment results.

[0013] The reconstructed model parameters are input into the validation dataset for validation. When the validation results meet the preset constraints, the reconstructed model parameters are determined as the optimized model parameters, and the optimized deep learning training model is output.

[0014] Optionally, the step of dividing the network layer parameters in the deep learning model to be optimized into parameter blocks and constructing a parameter block state set includes:

[0015] The parameters of the network layer in the deep learning model to be optimized are partitioned according to the network layer number, parameter tensor type, and parameter dimension range to generate corresponding parameter blocks;

[0016] Read the parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density for each parameter block.

[0017] Generate the corresponding parameter block state vector based on the parameter block number, network layer number, parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density.

[0018] All parameter block state vectors are combined to form a parameter block state set.

[0019] Optionally, the construction of the generalizable manifold shell domain includes:

[0020] The complete parameter snapshot, parameter block state, validation loss, validation accuracy, boundary sample confidence, easily confused sample class offset, and parameter energy density of each training round are cached in the order of training rounds to generate a time-ordered training state sequence.

[0021] In the training state sequence, the validation loss and parameter energy density of adjacent training rounds are compared in turn. If the validation loss of the current round is a local minimum and the parameter energy density is lower than that of the two rounds before and after, then all parameter blocks of the round are marked as stable kernel blocks, and the set of stable kernel blocks corresponding to the same training round is recorded as a set of stable kernels.

[0022] Between any two adjacent stable kernel points, the parameter block states are traversed along the training round direction. When a parameter block satisfies the following conditions in consecutive rounds: the verification loss fluctuation does not exceed the set tolerance, the parameter update magnitude remains non-zero, and the gradient direction is consistent, the parameter block is marked as a generalization transition shell block. All parameter blocks that meet the conditions constitute a set of generalization transition shell blocks.

[0023] Continue traversing the parameter block state along the training round direction. When the validation loss exceeds the set increase threshold, the confidence of the boundary sample decreases and the easily confused category shift increases, the parameter block that exceeds the coverage of the generalization transition shell block in the corresponding round is marked as the generalization collapse boundary block. All parameter blocks constitute the generalization collapse boundary block set.

[0024] Using the same parameter block number as anchor, stable core block, generalized transition shell block and generalized collapse boundary block with the same number are combined into shell domain ternary units. Unidirectional reachable continuation edges are established between shell domain ternary units based on the network layer adjacency relationship, temporal continuity relationship and gradient direction consistency relationship. After eliminating non-monotonic energy density paths, the continuous three-layer shell-like region enclosed by the shell domain ternary units and reachable continuation edges is determined as the reachable generalized manifold shell domain. According to the level of each parameter block in the shell domain, it is mapped to the stable core region, the generalized transition shell region and the generalized collapse boundary region respectively.

[0025] Optionally, the step of generating the shell domain assignment results of each parameter block in the reachable generalized manifold shell domain through dynamic mixing processing includes:

[0026] Read the parameter block number, network layer number, parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density corresponding to each parameter block, and generate parameter block label objects according to the field order;

[0027] Read the stable kernel block, generalized transition shell block and generalized collapse boundary block in the reachable generalized manifold shell domain, generate stable kernel point labeled objects, transition shell labeled objects and collapse boundary labeled objects respectively, and combine all labeled objects into a shell domain labeled sequence according to the training round order.

[0028] The shell domain labeled sequence is input into the improved HyperMixer model. The improved HyperMixer model consists of a label encoding layer, a position index embedding layer, a layer normalization layer, a hypernetwork weight generation layer, a dynamic token-mixing layer, and an output mapping layer.

[0029] The label encoding layer performs linear projection on each label object in the shell domain label sequence according to the field order, mapping the vector after concatenating the original fields into a 128-dimensional representation vector;

[0030] Location index embedding layer: Adds location information from the shell domain label sequence to each representation vector, and adds the location information vector to the representation vector element by element;

[0031] Layer normalization layer: The representation vector after adding position information is subjected to mean zeroing and variance normalization to obtain a normalized vector;

[0032] Supernetwork weight generation layer: Receives a normalized vector and combines it with the embedding vector and parameter energy density of the current training stage, and generates dynamic mixed weights through a two-level fully connected mapping.

[0033] Dynamic token-mixing layer: Four parallel mixing heads are set up. Each mixing head performs channel mixing and label mixing operations on parameter block marker objects, stable kernel point marker objects, transition shell marker objects and collapsed boundary marker objects respectively according to dynamic mixing weights, and outputs the mixed parameter block representation vector.

[0034] Output mapping layer: Performs fully connected mapping on the mixed parameter block representation vector, and outputs the stable kernel region matching score, the generalized transition shell region matching score, the generalized collapse boundary region matching score, and the shell domain deviation region matching score;

[0035] The matching scores of the stable core region, the generalized transition shell region, the generalized collapse boundary region, and the shell domain deviation region corresponding to each parameter block are sorted, and the region corresponding to the maximum matching score is determined as the shell domain assignment result of the parameter block in the reachable generalized manifold shell domain.

[0036] Output the shell domain attribution results corresponding to each parameter block and write them into the parameter block state set.

[0037] Optionally, determining that the current model parameters have a risk of detachment from the reachable generalization manifold shell domain includes:

[0038] Read the model parameter snapshots of the deep learning model to be optimized in the order of training rounds, and expand the parameter values ​​of the same parameter block in each training round into a parameter block vector according to the network layer number, parameter block number and parameter dimension.

[0039] For two adjacent training rounds of the same parameter block, perform bitwise interpolation processing, subtract the parameter block vector of the previous training round from the parameter block vector of the later training round, and obtain the parameter increment vector of the parameter block between the corresponding training rounds. Store all parameter increment vectors in the order of training rounds.

[0040] Connect the parameter increment vectors of the same parameter block according to the training rounds, set the starting point of each parameter increment vector to the parameter position of the previous training round, and set the ending point to the parameter position of the next training round to generate the parameter transition trajectory of the parameter block.

[0041] The direction change of two adjacent parameter increment vectors in the parameter transition trajectory is calculated. First, the length and direction sign of the two parameter increment vectors are read, then the direction angle between the two parameter increment vectors is calculated, and the direction angle is converted into radians to obtain the radians of the parameter transition trajectory of the parameter block in the corresponding training round.

[0042] The parameter transition trajectory radii of the same parameter block are continuously compared in the order of training rounds. When the parameter transition trajectory radii of the same parameter block increase sequentially in three consecutive training rounds, and the shell domain assignment result corresponding to the parameter block is a generalization collapse boundary region or an off-shell region, the parameter block is marked as a deviance risk parameter block. Based on the parameter block number, network layer number, training round number, parameter transition trajectory, parameter transition trajectory radii, and shell domain assignment result of the deviance risk parameter block, a deviance risk result for the reachable generalization manifold shell domain is generated.

[0043] Optionally, the step of projecting model parameters at risk of deviance onto a locally stable spanned subspace and generating reconstructed model parameters based on the shell domain attribution results includes:

[0044] In response to the detection of detachment risk from the reachable generalized manifold shell domain, the parameter block number, network layer number, training round number, and shell domain assignment result of the detachment risk parameter block are read, and stable kernel point blocks with the same parameter block number as the detachment risk parameter block are extracted from the stable kernel region of the reachable generalized manifold shell domain.

[0045] The stable kernel blocks are sorted according to the order of verification loss from low to high, parameter energy density from low to high, and training epoch from recent to distant. Stable kernel blocks that meet the preset selection quantity and are not connected to the generalization collapse boundary region are retained to generate a stable kernel candidate set.

[0046] The stable kernel block with the lowest verification loss in the candidate set of stable kernel points is used as the local reference block. The remaining stable kernel blocks are subjected to parameter difference processing with the local reference block to generate stable support directions. Stable support directions that point to the generalized collapse boundary region or the deviation region outside the shell domain are eliminated to generate the anti-collapse stable support set.

[0047] Based on the adjacency relationship of the network layer, the consistency relationship of parameter block numbering, and the continuity relationship of stable support direction, the stable support directions in the anti-collapse stable support set are combined into a local stable support skeleton, and the parameter activity range jointly defined by the local reference block, the candidate set of stable core points, and the local stable support skeleton is determined as the local stable tensor subspace.

[0048] Model parameters at risk of deviating are projected into a locally stable tensor subspace. Based on the shell domain assignment results, the parameter blocks in the stable core region are updated by maintaining their position, the parameter blocks in the generalized transition shell region are updated by limiting their amplitude, the parameter blocks in the generalized collapse boundary region are updated by projection rollback, and the parameter blocks in the off-topic region outside the shell domain are updated by low-rank reconstruction. The corresponding network layer parameter positions are written back according to the parameter block number to generate the reconstructed model parameters.

[0049] Optionally, the output optimized deep learning training model includes:

[0050] Read the validation loss, training loss, boundary sample confidence, easily confused sample class offset, and number of out-of-shell deviation parameter blocks corresponding to the model parameters before reconstruction, and generate benchmark validation records.

[0051] The reconstructed model parameters are written into the model to be validated, and forward validation is performed on the model to be validated using the validation dataset to generate a reconstruction validation record. The reconstruction validation record includes the reconstruction validation loss, the confidence of the reconstruction boundary samples, the class offset of the reconstructed easily confused samples, and the number of out-of-shell deviation parameter blocks.

[0052] The training loss is reviewed using the training dataset to generate a reconstructed training loss. The reconstructed training loss is then compared with the training loss in the benchmark validation record to obtain the training loss change results.

[0053] When the reconstructed validation loss is lower than the validation loss in the benchmark validation record, the confidence of the reconstructed boundary samples is not lower than the confidence of the boundary samples in the benchmark validation record, the class offset of the reconstructed easily confused samples is not greater than the class offset of the easily confused samples in the benchmark validation record, the number of reconstructed out-of-shell deviation parameter blocks is less than the number of out-of-shell deviation parameter blocks in the benchmark validation record, and the change in training loss does not exceed the preset training bounce threshold, the reconstructed model parameters are determined as the optimized model parameters.

[0054] Based on the optimized model parameters, continue to perform model training. Repeat the parameter block state update, shell domain attribution judgment, detachment risk detection and parameter reconstruction verification in consecutive training rounds. When no detachment risk result of reachable generalized manifold shell domain is generated within a consecutive preset number of rounds, and the verification loss change meets the preset stability condition, output the optimized deep learning training model.

[0055] The beneficial effects of this invention are:

[0056] This invention divides the network layer parameters of the deep learning model to be optimized into parameter blocks and constructs an reachable generalization manifold shell by combining stable kernel point parameters, generalization transition shell states, and generalization collapse boundary states. This allows the model training process to no longer rely solely on overall metrics such as training loss, validation loss, or accuracy for judgment, but to perform fine-grained identification of the generalization state of different network layer parameter blocks at the parameter space level. This enables timely detection of the hidden risks of parameter blocks migrating from the stable kernel region to the generalization collapse boundary region or the deviation region outside the shell.

[0057] This invention improves the HyperMixer model by dynamically mixing parameter block markers, stable kernel markers, transition shell markers, and collapsed boundary markers. This generates shell domain assignment results for each parameter block within the reachable generalization manifold shell domain. Furthermore, it combines the radian of the parameter transition trajectory to determine whether there is a risk of parameter deviation in the current model. Therefore, it can identify anomalous parameter migrations early, even when the training loss is still decreasing but the model's generalization ability has already begun to deteriorate. This avoids the problem of existing technologies only being able to passively stop early or perform overall parameter tuning after a significant performance degradation.

[0058] This invention, upon detecting the risk of detachment from the reachable generalization manifold shell region, constructs a locally stable tensor subspace based on stable kernel point parameters. Model parameters at risk of detachment are then projected into this subspace for topological rollback reconstruction. This allows parameter blocks deviating from the generalization region to be pulled back into the stable reachable range, while preserving effective training results within the stable kernel region and the generalization transition shell region. This invention reduces invalid rollbacks and excessive freezing, improving model training stability, parameter update controllability, and model generalization performance. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0060] Figure 1 This is a flowchart of a deep learning-based training model optimization method proposed in this invention;

[0061] Figure 2 This is a schematic diagram illustrating the process of constructing an reachable generalized manifold shell domain using a deep learning-based training model optimization method proposed in this invention. Detailed Implementation

[0062] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0063] refer to Figure 1 and Figure 2 A deep learning-based training model optimization method includes:

[0064] Read the training dataset, validation dataset, and deep learning model to be optimized; divide the network layer parameters in the deep learning model to be optimized into parameter blocks; and construct a set of parameter block states.

[0065] During model training, the stable kernel point parameters corresponding to the local optimal validation performance, the generalization transition shell state corresponding to the stable validation loss, and the generalization collapse boundary state corresponding to the increase in validation loss are recorded to construct an reachable generalization manifold shell domain.

[0066] Each parameter block, stable kernel point parameter, generalized transition shell state and generalized collapse boundary state are constructed as corresponding labeled objects and input into the improved HyperMixer model. The shell domain assignment results of each parameter block in the reachable generalized manifold shell domain are generated through dynamic mixing processing.

[0067] In consecutive training rounds, parameter transition trajectories are generated based on parameter increment vectors. The radian of the parameter transition trajectory is generated based on the directional changes between consecutive parameter increment vectors. When the radian of the parameter transition trajectory increases continuously and the corresponding shell domain assignment result is a generalized collapse boundary region or a shell domain deviation region, it is determined that the current model parameters have a risk of detachment from the reachable generalized manifold shell domain.

[0068] In response to the detection of the risk of detachment from the shell domain of the reachable generalized manifold, a locally stable tensor subspace is constructed based on the stable kernel point parameters. The model parameters with the risk of detachment are projected onto the locally stable tensor subspace, and the reconstructed model parameters are generated based on the shell domain assignment results.

[0069] The reconstructed model parameters are input into the validation dataset for validation. When the validation results meet the preset constraints, the reconstructed model parameters are determined as the optimized model parameters, and the optimized deep learning training model is output.

[0070] In this embodiment, the step of dividing the network layer parameters in the deep learning model to be optimized into parameter blocks and constructing a parameter block state set includes:

[0071] The parameters of the network layer in the deep learning model to be optimized are partitioned according to the network layer number, parameter tensor type, and parameter dimension range to generate corresponding parameter blocks;

[0072] Read the parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density for each parameter block.

[0073] Generate the corresponding parameter block state vector based on the parameter block number, network layer number, parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density.

[0074] All parameter block state vectors are combined to form a parameter block state set.

[0075] In this embodiment, the construction of the generalizable manifold shell domain includes:

[0076] The system caches complete parameter snapshots, parameter block states, validation loss, validation accuracy, boundary sample confidence, easily confused sample class offsets, and parameter energy density for each training epoch in chronological order, generating a time-ordered training state sequence. Specifically, the generation of the time-ordered training state sequence is as follows:

[0077] After reading the current training epoch number, first, according to the network layer number from shallow to deep and the parameter block number from small to large, expand all parameter block vectors in the complete parameter snapshot and write them into the parameter segment. Following the same order as the parameter segment, write the gradient direction change, parameter update magnitude, and parameter norm corresponding to each parameter block into the gradient segment. Then, write the validation loss, validation accuracy, boundary sample confidence, easily confused sample class offset, and parameter energy density of the current epoch into the performance segment. Finally, concatenate the epoch number, parameter segment, gradient segment, and performance segment in field order to generate a single training state vector. Append the training state vector to the training state matrix in ascending order of epoch number. Generate a four-byte timestamp index mapping list on the left side of the matrix so that any epoch number can directly locate the corresponding training state vector, resulting in a time-ordered training state sequence.

[0078] In the training state sequence, the validation loss and parameter energy density of adjacent training rounds are compared in turn. If the validation loss of the current round is a local minimum and the parameter energy density is lower than that of the two rounds before and after, then all parameter blocks of the round are marked as stable kernel blocks, and the set of stable kernel blocks corresponding to the same training round is recorded as a set of stable kernels.

[0079] Between any two adjacent stable kernel points, the parameter block states are traversed along the training round direction. When a parameter block satisfies the following conditions in consecutive rounds: the validation loss fluctuation does not exceed the set tolerance, the parameter update magnitude remains non-zero, and the gradient direction is consistent, the parameter block is marked as a generalization transition shell block. All parameter blocks that meet the conditions constitute a set of generalization transition shell blocks. The set tolerance is one-hundredth of the validation loss value of the previous training round. If the difference between the validation loss of the current training round and the validation loss of the previous training round does not exceed one-hundredth of the validation loss value of the previous training round, it is considered that the validation loss fluctuation has not exceeded the limit.

[0080] Continue traversing the parameter block state along the training round direction. When the validation loss exceeds the set rise threshold, the confidence of the boundary sample decreases and the easily confused class shift increases, the parameter block that exceeds the coverage of the generalization transition shell block in the corresponding round is marked as the generalization collapse boundary block. All parameter blocks constitute the generalization collapse boundary block set, where the set rise threshold is two percent of the validation loss value of the previous training round.

[0081] Using the same parameter block number as anchor, stable core blocks, generalized transition shell blocks, and generalized collapse boundary blocks with the same number are grouped into shell domain ternary units. Unidirectional reachable continuation edges are established between these shell domain ternary units based on network layer adjacency, temporal continuity, and gradient direction consistency. After eliminating non-monotonic paths based on energy density, the continuous three-layer shell-like region enclosed by the shell domain ternary units and reachable continuation edges is defined as the reachable generalized manifold shell domain. These are mapped to stable core regions, generalized transition shell regions, and generalized collapse boundary regions according to the level of each parameter block within the shell domain. Specifically:

[0082] Stable kernel region mapping: For each shell domain ternary unit, read the stable kernel point block, record the training round in which the stable kernel point block is located as the zero layer, search forward and backward along the unidirectional reachable continuation edge, and when the energy density of each step on the traversal path remains unchanged and the gradient direction sign does not change, directly map all parameter blocks in the current ternary unit as stable kernel regions.

[0083] Generalized transition shell mapping: After the stable core region is mapped, a breadth-first search is performed on the remaining ternary units in chronological order. When the search path satisfies the condition that the energy density increases but does not exceed 5% of the average energy density of the two consecutive rounds and the gradient direction sign is consistent with the previous hop, the parameter block in the corresponding ternary unit is mapped to the generalized transition shell region.

[0084] Generalized collapse boundary region mapping: Perform residual traversal on unmapped ternary units. When any reachable path has an energy density that increases by more than 5% of the average energy density of two consecutive rounds in a single hop or the gradient direction sign is flipped, it is determined that the path has entered the collapse region. All parameter blocks in the ternary units on the path are mapped to the generalized collapse boundary region.

[0085] After the mapping is completed, the final hierarchical label of each parameter block is recorded, and the hierarchical index is written into the parameter block state set to complete the hierarchical mapping of the stable core region, the generalized transition shell region, and the generalized collapse boundary region.

[0086] In this embodiment, the step of generating the shell domain assignment results of each parameter block in the reachable generalized manifold shell domain through dynamic mixing processing includes:

[0087] Read the parameter block number, network layer number, parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density corresponding to each parameter block, and generate parameter block label objects according to the field order;

[0088] Read the stable kernel block, generalized transition shell block, and generalized collapsed boundary block from the reachable generalized manifold shell domain. Generate stable kernel point labeled objects, transition shell labeled objects, and collapsed boundary labeled objects, respectively. Combine all labeled objects into a shell domain labeled sequence according to the training round order, wherein stable kernel point labeled objects, transition shell labeled objects, and collapsed boundary labeled objects are generated, specifically as follows:

[0089] Write the parameter block number of the stable kernel block into a two-byte unsigned integer field, write the network layer number into a two-byte unsigned integer field, write the training round number into a four-byte integer field, write the layer label kernel into a single-byte character field, and write the parameter energy density into a four-byte floating-point field. Concatenate the five fields in sequence to generate a 12-byte stable kernel label object.

[0090] Write the parameter block number, network layer number, training round number, hierarchical label shell, and parameter energy density of the generalized transition shell block in the same field order as the stable kernel point label object to generate a 12-byte transition shell label object.

[0091] Write the parameter block number, network layer number, training round number, hierarchical label edge, and parameter energy density of the generalized collapsed boundary block in the same field order to generate a 12-byte collapsed boundary label object;

[0092] According to the training round number in ascending order, and the network layer number in ascending order within the same round, all stable core point markers, transition shell markers, and collapse boundary markers of the current round are appended to the byte buffer in sequence. When the round number increases, a four-byte round separator 0xFFFF FFFF is inserted into the buffer. After traversing all rounds, an ordered shell domain marker sequence is obtained.

[0093] The shell domain labeled sequence is input into the improved HyperMixer model. The improved HyperMixer model consists of a label encoding layer, a position index embedding layer, a layer normalization layer, a hypernetwork weight generation layer, a dynamic token-mixing layer, and an output mapping layer.

[0094] The label encoding layer performs linear projection on each labeled object in the shell domain label sequence according to the field order, mapping the vector concatenated from the original fields to a 128-dimensional representation vector. Specifically, the mapping of the vector concatenated from the original fields to a 128-dimensional representation vector is as follows:

[0095] Five fields are extracted from a single shell domain label object: a two-byte unsigned integer parameter block number, a two-byte unsigned integer network layer number, a four-byte integer training epoch number, a single-byte character hierarchical label, and a four-byte floating-point parameter energy density. The twelve bytes are written to the local cache in byte order (highest byte first, lowest byte last). The two-byte integer field is then expanded to decimal and converted to a 32-bit floating-point format. The single-byte character field is mapped to a floating-point label value between zero and three. The remaining four-byte integer and floating-point fields maintain their original floating-point precision. The five floating-point numbers are merged into a five-bit floating-point vector in the original field order. This vector is multiplied by a five-row, 128-column first mapping weight matrix and a 128-bit first mapping bias vector is added. Both the weight matrix and bias vector are randomly and uniformly assigned values ​​during initialization and updated end-to-end during model training. After mapping, a 128-bit representation vector is obtained.

[0096] Location index embedding layer: Adds location information from the shell domain label sequence to each representation vector, and adds the location information vector to the representation vector element by element;

[0097] Layer normalization layer: The representation vector after adding position information is subjected to mean zeroing and variance normalization to obtain a normalized vector;

[0098] The supernetwork weight generation layer receives a normalized vector and combines it with the embedding vector and parameter energy density from the current training phase. It then generates dynamically mixed weights through a two-level fully connected mapping. Specifically, the generation of dynamically mixed weights through this two-level fully connected mapping is as follows:

[0099] The 128-dimensional normalized vector, the 64-dimensional training stage embedding vector, and the single-dimensional parameter energy density floating-point number are sequentially concatenated into a 193-dimensional input vector. The first-level fully connected mapping is performed, multiplying the input vector by a first weight matrix of size 193 rows and 64 columns and adding a first bias vector of length 64 to obtain a 64-dimensional latent vector. A linear rectified function with a threshold of 0.1 is applied to each element of the latent vector. The second-level fully connected mapping is performed, multiplying the 64-dimensional latent vector by a second weight matrix of size 64 rows and 32 columns and adding a second bias vector of length 32 to obtain a 32-dimensional output vector. The 32-dimensional output vector is then evenly divided into segments of four based on the number of parallel mixing heads. Each segment contains eight floating-point numbers corresponding to the channel mixing weight and the tag mixing weight fields, which are then written to the weight buffer of the dynamic token-mixing layer.

[0100] Dynamic token-mixing layer: Four parallel mixing heads are set up. Each mixing head performs channel mixing and token mixing operations on parameter block marker objects, stable kernel point marker objects, transition shell marker objects, and collapsed boundary marker objects respectively, based on dynamic mixing weights. The mixed parameter block representation vector is output. Specifically, the mixed parameter block representation vector is as follows:

[0101] The parameter block marker object, along with four 128-dimensional representation vectors (stable core point marker object, transition shell marker object, and collapsed boundary marker object with the same number), are concatenated column-wise into a stacked matrix of size 128 rows and 4 columns. Four parallel mixing heads receive channel mixing weights and label mixing weights from the supernetwork weight generation layer, respectively. Each mixing head first performs channel mixing on the stacked matrix in the column direction, calculating a weighted sum of the four elements in the same row according to the channel mixing weights to obtain a 128-dimensional channel mixing vector. Then, label mixing is performed on the channel mixing vector in the row direction, dividing the vector into four segments of 32 elements each. The elements within each segment are weighted according to their position based on the label mixing weights and then summed to generate 32-dimensional label mixing segments. The four label mixing segments output by the four mixing heads are concatenated in their original order to form a 128-dimensional mixing vector. Layer normalization and residual connection are performed on the 128-dimensional mixing vector, which is then added to the original parameter block representation vector to obtain the final 128-dimensional mixed parameter block representation vector used for output mapping layer processing.

[0102] Output Mapping Layer: Performs fully connected mapping on the mixed parameter block representation vector, outputting the matching scores for the stable kernel region, the generalized transition shell region, the generalized collapsed boundary region, and the off-shell region. Specifically, the matching scores for the stable kernel region, the generalized transition shell region, the generalized collapsed boundary region, and the off-shell region are as follows:

[0103] A 128-dimensional representation vector is input into a 128x4 weight matrix and added to the corresponding 4-dimensional bias vector to generate a 4-dimensional linear mapping result. A normalized activation with a temperature coefficient of 1.0 is performed on the 4-dimensional linear mapping result. The value of the dimension containing the maximum value is subtracted from the mean of the 4-dimensional vector and divided by the square root of the vector variance to obtain a 4-dimensional normalized output vector. An exponential mapping is then performed on the normalized output vector, and the four components are summed. Each component is divided by the summation value to obtain a 4-dimensional normalized probability vector, where the first component serves as the stable core region matching score, the second component as the generalized transition shell region matching score, the third component as the generalized collapse boundary region matching score, and the fourth component as the shell domain deviation region matching score. The 4-dimensional normalized probability vector is then written into the shell domain matching score field corresponding to the parameter block.

[0104] The matching scores of the stable core region, the generalized transition shell region, the generalized collapse boundary region, and the shell domain deviation region corresponding to each parameter block are sorted, and the region corresponding to the maximum matching score is determined as the shell domain assignment result of the parameter block in the reachable generalized manifold shell domain.

[0105] Output the shell domain attribution results corresponding to each parameter block and write them into the parameter block state set;

[0106] The tag coding layer includes:

[0107] Field splitting unit: Split the original labeled object according to the field order of parameter block number, network layer number, training round number, layer label and parameter energy density;

[0108] Numerical expansion unit: expands the split integer field into a 32-bit floating-point format;

[0109] Tag mapping unit: maps hierarchical tags to fixed floating-point tag values;

[0110] Field concatenation unit: Caches all floating-point fields in field order;

[0111] Linear projection unit: performs dimension mapping on the concatenated field vector;

[0112] After receiving the original labeled objects from the shell domain labeled sequence, the label coding layer first splits the parameter block number, network layer number, training epoch number, hierarchical label, and parameter energy density into a fixed byte length. The numerical expansion unit expands the parameter block number, network layer number, and training epoch number into floating-point format. The label mapping unit maps the kernel, shell, and edge to 0, 1, and 2, respectively. The field concatenation unit concatenates all floating-point fields into an input vector of length 5 according to the field order. The linear projection unit performs a linear mapping on the input vector using a 5×128-dimensional projection matrix and outputs a 128-dimensional label representation vector.

[0113] The location index embedding layer includes:

[0114] Round position generation unit: Generates time position indexes based on training rounds;

[0115] Hierarchical position generation unit: Generates hierarchical position index based on network layer number;

[0116] Position encoding unit: caches the position encoding vectors corresponding to different positions;

[0117] Location fusion unit: performs fusion of location encoding vector and label representation vector;

[0118] After receiving the 128-dimensional labeled representation vector, the position index embedding layer generates a time position index according to the training round number, and the hierarchical position generation unit generates a hierarchical position index according to the network layer number. The position encoding unit pre-stores a time position encoding vector and a hierarchical position encoding vector with a length of 128. The position fusion unit adds the time position encoding vector and the hierarchical position encoding vector to the corresponding labeled representation vector element by element to generate a position fusion vector containing time position and hierarchical position information.

[0119] The layer normalization layer includes:

[0120] Mean Statistical Unit: The mean of all elements in the position fusion vector;

[0121] Variance statistics unit: Statistically calculates the variance of all elements in the position fusion vector;

[0122] Normalization calculation unit: Performs normalization based on mean and variance;

[0123] Scaling and Offset Unit: Performs scaling and offset on the normalized result;

[0124] After receiving the position fusion vector, the normalization layer calculates the average of 128 elements using the mean statistics unit; the variance statistics unit calculates the mean of the squared differences between all elements and the mean; the normalization calculation unit subtracts the mean from each element and divides it by the square root of the variance; the scaling and offset unit multiplies the normalization result by a trainable scaling factor and adds an offset factor to generate a normalized vector.

[0125] The supernet weight generation layer includes:

[0126] Stage Embedding Unit: Caches the embedding vector of the current training stage;

[0127] Energy density splicing unit: Writes the parameter energy density to the input tail;

[0128] First-level fully connected mapping unit: performs the first dimensionality compression on the input vector;

[0129] Activate the transform unit: Perform linear rectification on the compression result;

[0130] Second-level fully connected mapping unit: generates dynamic blending weights;

[0131] Weight distribution unit: Caches dynamically mixed weights according to the mixing header number;

[0132] After receiving the normalized vector, the supernetwork weight generation layer reads the 64-dimensional training stage embedding vector by the stage embedding unit; the energy density concatenation unit appends the parameter energy density to the end of the normalized vector to form a joint input vector; the first-level fully connected mapping unit maps the joint input vector to a 64-dimensional latent vector; the activation transformation unit sets the elements less than 0 in the latent vector to zero; the second-level fully connected mapping unit maps the 64-dimensional latent vector to a 32-dimensional dynamic weight vector; the weight distribution unit divides the 32-dimensional dynamic weight vector into four groups, with each group containing eight weight values ​​corresponding to four parallel mixing heads.

[0133] The dynamic token-mixing layer includes:

[0134] Marking stacking units: combining parameter block marker objects with three types of shell domain reference marker objects into a stacking matrix;

[0135] Channel blending unit: performs column-direction weighted blending on the stacked matrix;

[0136] Mark the blending unit: Perform row-direction rearrangement blending on the channel blending results;

[0137] Hybrid Head Buffer Unit: Buffers the output results of 4 parallel hybrid heads;

[0138] Residual connection unit: performs residual superposition of the mixed result with the original input;

[0139] After receiving the dynamic mixing weights, the dynamic token-mixing layer concatenates the parameter block marker objects, stable kernel point marker objects, transition shell marker objects, and collapsed boundary marker objects into a 128×4 stacked matrix. The channel mixing unit performs a weighted summation on the four elements in the same row according to the channel weights of the corresponding mixing head, generating a 128-dimensional channel mixing vector. The marker mixing unit divides the channel mixing vector into four segments, each with 32 elements, and performs intra-segment rearrangement and weighted mixing according to the marker mixing weights. The mixing head caching unit stores the output results of the four parallel mixing heads. The residual connection unit adds the concatenated 128-dimensional mixing result to the original parameter block representation vector element by element, generating the mixed parameter block representation vector.

[0140] The output mapping layer includes:

[0141] Fully connected classification unit: performs linear classification on the mixed parameter block representation vector;

[0142] Normalization activation unit: Performs normalization on the classification results;

[0143] Probability allocation unit: generates four types of shell domain matching scores;

[0144] Attribution determination unit: Outputs the final shell domain attribution result;

[0145] After receiving the mixed parameter block representation vector, the output mapping layer maps the 128-dimensional vector into a 4-dimensional classification vector; the normalization activation unit performs normalization processing on the 4-dimensional classification vector; the probability allocation unit generates the stable kernel region matching score, the generalized transition shell region matching score, the generalized collapse boundary region matching score, and the shell domain deviation region matching score; the attribution determination unit reads the maximum value among the four matching scores and determines the corresponding region as the final shell domain attribution result of the parameter block in the reachable generalized manifold shell domain.

[0146] In this embodiment, determining that the current model parameters have a risk of detachment from the reachable generalization manifold shell domain includes:

[0147] Read the model parameter snapshots of the deep learning model to be optimized in the order of training rounds, and expand the parameter values ​​of the same parameter block in each training round into a parameter block vector according to the network layer number, parameter block number and parameter dimension.

[0148] For two adjacent training rounds of the same parameter block, perform bitwise interpolation processing, subtract the parameter block vector of the previous training round from the parameter block vector of the later training round, and obtain the parameter increment vector of the parameter block between the corresponding training rounds. Store all parameter increment vectors in the order of training rounds.

[0149] The parameter increment vectors of the same parameter block are connected in the order of training rounds. The starting point of each parameter increment vector is set to the parameter position of the previous training round, and the ending point is set to the parameter position of the next training round, thus generating the parameter transition trajectory of the parameter block. Specifically, the parameter transition trajectory of the parameter block is generated as follows:

[0150] The parameter increment vectors of the same parameter block in consecutive training rounds are written into the trajectory cache in ascending order of round number. The trajectory cache is then processed as follows:

[0151] Write the round-one parameter position as the first node in the trajectory cache header. Read the two adjacent parameter increment vectors in sequence, set the parameter position corresponding to the first element of the currently read parameter increment vector as the start point, and set the parameter position corresponding to the last element as the end point. Create a new directed segment record in the trajectory list, write the start point, end point, and the training round number to the segment record, and append the segment record to the trajectory list. After traversal, perform integrity verification on the trajectory list. If the end point of any two adjacent segment records is inconsistent with the start point of the next segment record, insert a link repair segment between them and set the start point of the repair segment to the previous end point and the end point to the next start point to ensure trajectory continuity.

[0152] Concatenate all segment records in the trajectory list in order into a parameter transition trajectory object specific to the parameter block, and write the object into the trajectory field corresponding to the parameter block state set;

[0153] The direction change of two adjacent parameter increment vectors in the parameter transition trajectory is calculated. First, the length and direction sign of the two parameter increment vectors are read, then the direction angle between the two parameter increment vectors is calculated, and the direction angle is converted into a radian value to obtain the radian value of the parameter transition trajectory of the parameter block in the corresponding training round. Specifically, the radian value of the parameter transition trajectory of the parameter block in the corresponding training round is as follows:

[0154] Create a length cache for the current computation window and calculate the length of each of the two parameter increment vectors. Square the elements of each dimension of each vector, sum them, and take the square root of the result to obtain the vector length. Generate a sequence of direction signs. If the value of each dimension of the current vector is greater than or equal to zero, write the sign "+"; otherwise, write the sign "-". Multiply the elements of each dimension of the two vectors and sum them to obtain a vector similarity scalar. Divide the scalar by the length of the previous vector and then by the length of the next vector to limit the result to between -1 and 1. Then, call the inverse cosine function locally to convert the ratio into a direction angle value. Divide the angle value into radians by the circumferential constant π divided by a conversion factor of 180. Write the radian value into the radian field of the parameter transition trajectory corresponding to the current training round, and append it to the trajectory radian list for storage.

[0155] The parameter transition trajectory radians of the same parameter block are continuously compared in the order of training epochs. When the parameter transition trajectory radians of the same parameter block increase sequentially in three consecutive training epochs, and the shell domain assignment result corresponding to the parameter block is a generalization collapse boundary region or an off-shell region, the parameter block is marked as a deviance risk parameter block. Based on the parameter block number, network layer number, training epoch number, parameter transition trajectory, parameter transition trajectory radian, and shell domain assignment result of the deviance risk parameter block, a deviance risk result for the reachable generalization manifold shell domain is generated. Specifically, the generation of the deviance risk result for the reachable generalization manifold shell domain is as follows:

[0156] A sliding window of length three is established according to the training rounds. The radian values ​​of the parameter transition trajectories corresponding to three consecutive training rounds of the same parameter block are written into the window register. The radian values ​​of the first, second, and third training rounds in the window register are compared sequentially. When the radian value of the third training round is greater than that of the second training round, and the radian value of the second training round is greater than that of the first training round, an increasing radian status label is written for the parameter block in the risk label register. The shell domain assignment result of the parameter block in the third training round is then read. When the shell domain assignment result is a generalization collapse boundary region or an off-shell region, in The risk marker register area is written with a shell domain out-of-bounds status label. When the risk marker register area contains both an increasing radian status label and a shell domain out-of-bounds status label, the parameter block number, network layer number, third training round number, corresponding parameter transition trajectory object, radian value of the parameter transition trajectory in the third training round, and shell domain attribution result in the third training round are concatenated into a risk record tuple, and the risk record tuple is appended to the risk detachment list. After all parameter blocks are traversed, the risk detachment list is written to the risk detachment result field of the reachable generalized manifold shell domain, and the risk detachment marker is recorded synchronously in the parameter block state set.

[0157] In this embodiment, projecting model parameters with escape risk onto a locally stable spanned subspace and generating reconstructed model parameters based on shell domain attribution results includes:

[0158] In response to the detection of detachment risk from the reachable generalized manifold shell domain, the parameter block number, network layer number, training round number, and shell domain assignment result of the detachment risk parameter block are read, and stable kernel point blocks with the same parameter block number as the detachment risk parameter block are extracted from the stable kernel region of the reachable generalized manifold shell domain.

[0159] The stable kernel blocks are sorted according to the order of verification loss from low to high, parameter energy density from low to high, and training epoch from recent to distant. Stable kernel blocks that meet the preset selection quantity and are not connected to the generalization collapse boundary region are retained to generate a stable kernel candidate set.

[0160] The stable kernel block with the lowest verification loss in the candidate set of stable kernel blocks is used as the local reference block. The remaining stable kernel blocks are then subjected to parameter difference processing with the local reference block to generate stable support directions. Stable support directions pointing towards the generalized collapse boundary region or the off-boundary region outside the shell domain are removed, generating an anti-collapse stable support set. Specifically, the generated stable support directions are as follows:

[0161] Read the parameter vectors of the current stable kernel block to be processed and the local reference block in the order of parameter dimensions, perform element-wise difference processing on the parameter values ​​at corresponding positions of the two vectors, and write the difference results into the difference buffer.

[0162] Traverse each difference element in the difference buffer, record positive difference as positive, negative difference as negative, and zero difference as undirected. Encode positive, negative, and undirected as 1, 0, and empty in binary form, respectively. Then concatenate all the encoded results in order to obtain a direction sign sequence.

[0163] The direction symbol sequence is compressed, and segments of the same symbol appearing three or more times consecutively are merged and recorded as a single direction signature. The ratio of forward segments to reverse segments in the direction signature is then calculated.

[0164] When the proportion of positive segments in the directional signature is higher than 70% and the difference between the network layer where the local reference block is located is no more than two layers, the directional signature is written into the stable support directional list; otherwise, the directional signature is removed. After all stable core block processing is completed, the stable support directional list is traversed, and directional signatures that point to the generalized collapse boundary region or the deviation region outside the shell domain are removed. The remaining directional signature set is the anti-collapse stable support set.

[0165] Based on the network layer adjacency relationship, parameter block number consistency relationship, and stable support direction continuity relationship, the stable support directions in the anti-collapse stable support set are combined into a local stable support skeleton. The parameter activity range jointly defined by the local reference block, the candidate set of stable core points, and the local stable support skeleton is determined as the local stability spanning subspace. Specifically, the combination of stable support directions in the anti-collapse stable support set into the local stable support skeleton is as follows:

[0166] First, for each stable support direction, read the starting parameter block number, ending parameter block number, and starting and ending network layer numbers, and write these two numbers into the direction edge table in the order of starting and ending. Sort the direction edge table in ascending order of the starting network layer number. In the sorting results, direction edges with adjacent starting network layer numbers and the same parameter block number are recorded as continuation edges in the same layer, and the starting parameter block number is written into the continuation linked list and marked as intra-layer continuation. Then, in the direction edge table, direction edges with a difference of one between the ending network layer number and the starting network layer number of the next direction edge, and whose ending parameter block number is the same as the starting parameter block number of the next direction edge, are recorded as cross-layer continuation edges. For each continuation edge, the termination parameter block number is written into the cross-layer linked list and the inter-layer continuation is marked. The same-layer continuation edges and cross-layer continuation edges are merged according to the node order recorded in the continuation linked list. After each merge, an acyclic directed line segment is generated and written into the skeleton segment buffer. A loop check is performed on the skeleton segment buffer. If the starting node and the ending node are found to be duplicated, the longest skeleton segment forming a closed loop is deleted and the breakpoint is reconnected. All acyclic skeleton segments are appended to the local stable support skeleton list in ascending order of the starting network layer number, forming a local stable support skeleton jointly defined by the local reference block, the stable core point candidate set, and the acyclic skeleton segments.

[0167] Model parameters at risk of deviating are projected into a locally stable tensor subspace. Based on the shell domain assignment results, the parameter blocks in the stable core region are updated with preservation, the parameter blocks in the generalized transition shell region are updated with amplitude limiting, the parameter blocks in the generalized collapse boundary region are updated with projection rollback, and the parameter blocks in the out-of-shell deviation region are updated with low-rank reconstruction. The corresponding network layer parameter positions are written back according to the parameter block number to generate the reconstructed model parameters. Specifically, the generated reconstructed model parameters are as follows:

[0168] The basis vectors corresponding to the skeleton segments are read sequentially from the locally stable support skeleton, and orthogonalization is performed once according to the network layer order. The processed basis vectors are written into the orthogonal basis list to form a basis vector group of the locally stable spanned subspace.

[0169] Read the detached risk parameter block vectors one by one from the projection buffer, calculate the scalar projection of the vectors on each basis vector in turn by referring to the orthogonal basis list, add all the scalar projection values ​​with the corresponding basis vectors in a vector weighted manner to obtain the projection rollback vector, and write the projection rollback vector into the rollback buffer.

[0170] Based on the shell domain assignment result of the parameter block, classify and process: when the parameter block is assigned to a stable kernel region, directly write the original parameter block vector into the output buffer;

[0171] When the parameter block belongs to the generalized transition shell region, the difference between the original parameter block vector and the rollback vector is read. If the absolute value of the difference exceeds the preset amplitude threshold, it is truncated in the threshold direction and written to the output buffer; otherwise, it is written to the original parameter block vector.

[0172] When the parameter block belongs to the generalization collapse boundary region, it is replaced with a rollback vector and written to the output buffer.

[0173] When the parameter block belongs to the off-bound region outside the shell domain, low-rank reconstruction is performed on the original parameter block vector. The vector is truncated in the main diagonal direction to the first few dimensions that account for 80% of the energy and then written to the output buffer.

[0174] After all parameter blocks have been processed, the vectors in the output buffer are written back to the model parameter table in their original order according to the parameter block number and network layer number index, overwriting the corresponding parameter positions, and finally generating the reconstructed model parameters.

[0175] In this embodiment, the output optimized deep learning training model includes:

[0176] Read the validation loss, training loss, boundary sample confidence, easily confused sample class offset, and number of out-of-shell deviation parameter blocks corresponding to the model parameters before reconstruction, and generate benchmark validation records.

[0177] The reconstructed model parameters are written into the model to be validated, and forward validation is performed on the model to be validated using the validation dataset to generate a reconstruction validation record. The reconstruction validation record includes the reconstruction validation loss, the confidence of the reconstruction boundary samples, the class offset of the reconstructed easily confused samples, and the number of out-of-shell deviation parameter blocks.

[0178] Using the training dataset, the training loss of the model to be validated is reviewed to generate a reconstructed training loss. This reconstructed training loss is then compared with the training loss in the benchmark validation record to obtain the changes in the training loss. Specifically, the generation of the reconstructed training loss involves:

[0179] In the inference engine, gradient calculation is disabled in inference mode, forward propagation is performed on the input tensor, and the model output vector is temporarily saved.

[0180] Call the loss calculation module to compare the model output vector with the corresponding label tensor one by one, select the cross-entropy or mean squared error loss function according to the task type, and write the single sample loss value obtained for each sample into the batch loss cache.

[0181] The average loss of all single samples in the batch loss cache is calculated by averaging the number of samples to obtain the average loss of the current batch. The average loss is then appended to the batch loss sequence. After iterating through all batches, the batch loss sequence is accumulated and divided by the number of batches to obtain the reconstructed training loss corresponding to the full training data. The reconstructed training loss is then written to the training loss field of the reconstructed verification record.

[0182] When the reconstructed validation loss is lower than the validation loss in the benchmark validation record, the confidence of the reconstructed boundary samples is not lower than the confidence of the boundary samples in the benchmark validation record, the class offset of the reconstructed easily confused samples is not greater than the class offset of the easily confused samples in the benchmark validation record, the number of reconstructed out-of-shell deviation parameter blocks is less than the number of out-of-shell deviation parameter blocks in the benchmark validation record, and the change in training loss does not exceed the preset training bounce threshold, the reconstructed model parameters are determined as the optimized model parameters.

[0183] Based on the optimized model parameters, continue to perform model training. Repeat the parameter block state update, shell domain attribution judgment, detachment risk detection and parameter reconstruction verification in consecutive training rounds. When no detachment risk result of reachable generalized manifold shell domain is generated within a consecutive preset number of rounds, and the verification loss change meets the preset stability condition, output the optimized deep learning training model.

[0184] Example 1: In a training task for an industrial defect image classification model, the system receives a batch of visually acquired samples from a production line, totaling 48,600 images. The resolution is uniformly adjusted to 512×512 pixels, including 31,200 normal samples, 8,200 scratch samples, 5,300 indentation samples, and 3,900 edge notch samples. The validation set contains 7,200 images, including 1,296 boundary defect samples, 792 reflective interference samples, and 864 texture similarity samples. The deep learning model to be optimized contains 12 trainable network layers, with an initial parameter count of 18.6M, a batch size of 64, an initial learning rate of 0.001, and a maximum training epoch of 80. In traditional training, the training loss decreased to 0.312 and the validation loss decreased to 0.421 in the 28th round. However, when the training loss continued to decrease to 0.276 in the 32nd round, the validation loss increased to 0.501, and the average confidence of the boundary samples decreased from 0.812 to 0.724. This indicates that the model has experienced a problem of decreasing training loss but deteriorating generalization performance.

[0185] After the data enters the method of this invention, the system first reads a complete parameter snapshot of each round, and divides the 12 network layers into 96 parameter blocks according to the network layer number from shallow to deep and the parameter block number from small to large. Taking the 14th parameter block of the 6th network layer as an example, in the 24th round, the parameter norm is 8.42, the parameter update magnitude is 0.018, the gradient direction change is 0.21, the training loss decreases by 0.021, the validation loss changes by -0.009, and the parameter energy density is 2.00; in the 31st round, the parameter norm of this parameter block rises to 9.18, the parameter update magnitude rises to 0.034, the training loss still decreases by 0.017, but the validation loss increases by 0.013, and the parameter energy density rises to 2.62. The system sequentially writes the 96 parameter block states, validation loss, validation accuracy, boundary sample confidence, and easily confused class offsets of each round into the training state matrix. The number of rows in the matrix increases with each training round, and a 32×96 set of parameter block state records is formed in the 32nd round.

[0186] During the construction of the reachable generalization manifold shell domain, the system traverses the training state matrix. The validation losses in rounds 18, 23, and 27 are 0.448, 0.419, and 0.407, respectively, and the parameter energy densities are 1.46, 1.39, and 1.34, respectively, all lower than those in adjacent rounds, and are therefore recorded as stable kernel point parameters. Between rounds 19 and 26, the validation loss fluctuates between 0.407 and 0.436, with the maximum fluctuation not exceeding 1% of the validation loss of the previous round. Among these, 61 parameter blocks maintain non-zero updates and have continuous and consistent gradient directions, and are marked as generalization transition shell blocks. From round 29 to round 34, the validation loss increased from 0.429 to 0.501, exceeding the 2% increase threshold of the previous round's validation loss. The confidence of the boundary samples decreased from 0.801 to 0.724, and the offset of easily confused categories increased from 0.064 to 0.118. The system marked 18 parameter blocks that exceeded the coverage of the transition shell as generalization collapse boundary blocks, and used the same parameter block number as anchors to generate 96 shell domain ternary units, forming a stable core region, a generalization transition shell region, and a generalization collapse boundary region.

[0187] The system constructs the current parameter block, stable kernel block, generalized transition shell block, and generalized collapse boundary block as labeled objects and inputs them into the improved HyperMixer model. Each labeled object contains five fields: parameter block number, network layer number, training epoch number, hierarchical label, and parameter energy density. The label encoding layer maps the 5-dimensional field vector to a 128-dimensional representation vector, the position index embedding layer adds the training epoch position and network layer position, and the layer normalization layer completes mean zeroing and variance normalization. The hypernetwork weight generation layer concatenates the 128-dimensional normalized vector, the 64-dimensional training stage embedding vector, and the 1-dimensional parameter energy density into a 193-dimensional input and generates 32-dimensional dynamic mixing weights. The dynamic token-mixing layer sets up four parallel mixing heads to combine the labeled objects of the parameter block, stable kernel, transition shell, and collapse boundary corresponding to the same parameter block into a 128×4 matrix for mixing. The output results of the 32nd round show that among the 96 parameter blocks, 51 belong to the stable core region, 27 belong to the generalized transition shell region, 13 belong to the generalized collapse boundary region, and 5 belong to the shell domain deviation region.

[0188] In parameter transition trajectory analysis, the system reads parameter snapshots from rounds 30 to 33. For the 14th parameter block of the 6th network layer, the system subtracts the 30th round vector from the 31st round vector to obtain the first parameter increment vector, subtracts the 31st round vector from the 32nd round vector to obtain the second parameter increment vector, and subtracts the 32nd round vector from the 33rd round vector to obtain the third parameter increment vector. These are then connected in round order to form the parameter transition trajectory. After calculating the direction change, the radians of the parameter transition trajectory for this parameter block in the three consecutive windows are 0.41, 0.67, and 0.93, respectively, showing a continuous increase. Simultaneously, the shell domain of this parameter block is classified as an out-of-shell deviation region, and the system adds it to the deviance risk list. A total of 18 deviance risk parameter blocks were identified in round 32, including 9 from the 6th to 8th network layers, 6 from the classification mapping layer, and 3 from the shallow feature extraction layer.

[0189] Upon detecting a risk of deviating from the target region, the system reads the corresponding stable kernel block from the stable kernel region. Taking the 14th parameter block of the 6th network layer as an example, the system selects stable kernel blocks from rounds 18, 23, and 27. Round 27 has the lowest verification loss and is set as the local reference block. The system performs element-wise difference between the remaining stable kernel blocks and the local reference block to generate stable support directions. After removing directions pointing to the generalization collapse boundary region and the off-shell region, four anti-collapse stable support directions are obtained. Subsequently, these directions are combined into a local stable support skeleton according to the network layer adjacency relationship and the consistency of parameter block numbers. The 18 parameter blocks with deviating from the target region are projected onto the local stable spanning subspace. The parameter blocks in the stable kernel region remain unchanged. The single-round update amplitude of the parameter blocks in the transition shell region is limited to 40% of the original amplitude. The parameter blocks in the collapse boundary region are replaced with projected rollback vectors. The parameter blocks in the off-shell region retain 80% of the principal energy and then undergo low-rank reconstruction.

[0190] After reconstruction, the system reloads and validates the model. Before the 33rd round of reconstruction, the training loss was 0.271 and the validation loss was 0.501. After reconstruction, the training loss was 0.279, a rebound of 2.95%, below the 5% constraint threshold. The validation loss decreased to 0.438, the average confidence of boundary samples increased from 0.724 to 0.789, the easily confused class offset decreased from 0.118 to 0.071, and the number of out-of-shell deviation parameter blocks decreased from 5 to 1. After continuing training to the 48th round, the method of this invention had a training loss of 0.236, a validation loss of 0.361, a validation accuracy of 91.8%, a boundary defect sample identification accuracy of 86.9%, and an easily confused sample misclassification rate of 6.8%. The traditional learning rate decay and early stopping method stopped at the 36th round, with a training loss of 0.248, a validation loss of 0.492, a validation accuracy of 86.7%, a boundary defect sample identification accuracy of 78.4%, and an easily confused sample misclassification rate of 12.6%. In the slightly chipped edge image of sample A, the traditional method outputs a confidence score of 0.62 for the normal category and 0.31 for the chipped edge category, while the present invention outputs a confidence score of 0.74 for the chipped edge category. In the reflective scratch image of sample B, the traditional method outputs a scratch confidence score of 0.54, while the present invention outputs 0.81. In the texture-similar indentation image of sample C, the traditional method misclassifies the error as normal at a rate of 14.3%, while the present invention reduces this to 7.1%.

[0191] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for optimizing a training model based on deep learning, characterized in that, include: Read the training dataset, validation dataset, and deep learning model to be optimized; divide the network layer parameters in the deep learning model to be optimized into parameter blocks; and construct a set of parameter block states. During model training, the stable kernel point parameters corresponding to the local optimal validation performance, the generalization transition shell state corresponding to the stable validation loss, and the generalization collapse boundary state corresponding to the increase in validation loss are recorded to construct an reachable generalization manifold shell domain. Each parameter block, stable kernel point parameter, generalized transition shell state and generalized collapse boundary state are constructed as corresponding labeled objects and input into the improved HyperMixer model. The shell domain assignment results of each parameter block in the reachable generalized manifold shell domain are generated through dynamic mixing processing. In consecutive training rounds, parameter transition trajectories are generated based on parameter increment vectors. The radian of the parameter transition trajectory is generated based on the directional changes between consecutive parameter increment vectors. When the radian of the parameter transition trajectory increases continuously and the corresponding shell domain assignment result is a generalized collapse boundary region or a shell domain deviation region, it is determined that the current model parameters have a risk of detachment from the reachable generalized manifold shell domain. In response to the detection of the risk of detachment from the shell domain of the reachable generalized manifold, a locally stable tensor subspace is constructed based on the stable kernel point parameters. The model parameters with the risk of detachment are projected onto the locally stable tensor subspace, and the reconstructed model parameters are generated based on the shell domain assignment results. The reconstructed model parameters are input into the validation dataset for validation. When the validation results meet the preset constraints, the reconstructed model parameters are determined as the optimized model parameters, and the optimized deep learning training model is output.

2. The method for optimizing a training model based on deep learning according to claim 1, characterized in that, The process of dividing the network layer parameters in the deep learning model to be optimized into parameter blocks and constructing a parameter block state set includes: The parameters of the network layer in the deep learning model to be optimized are partitioned according to the network layer number, parameter tensor type, and parameter dimension range to generate corresponding parameter blocks; Read the parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density for each parameter block. Generate the corresponding parameter block state vector based on the parameter block number, network layer number, parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density. All parameter block state vectors are combined to form a parameter block state set.

3. The method for optimizing a training model based on deep learning according to claim 1, characterized in that, The construction of the generalizable manifold shell domain includes: The complete parameter snapshot, parameter block state, validation loss, validation accuracy, boundary sample confidence, easily confused sample class offset, and parameter energy density of each training round are cached in the order of training rounds to generate a time-ordered training state sequence. In the training state sequence, the validation loss and parameter energy density of adjacent training rounds are compared in turn. If the validation loss of the current round is a local minimum and the parameter energy density is lower than that of the two rounds before and after, then all parameter blocks of the round are marked as stable kernel blocks, and the set of stable kernel blocks corresponding to the same training round is recorded as a set of stable kernels. Between any two adjacent stable kernel points, the parameter block states are traversed along the training round direction. When a parameter block satisfies the following conditions in consecutive rounds: the verification loss fluctuation does not exceed the set tolerance, the parameter update magnitude remains non-zero, and the gradient direction is consistent, the parameter block is marked as a generalization transition shell block. All parameter blocks that meet the conditions constitute a set of generalization transition shell blocks. Continue traversing the parameter block state along the training round direction. When the validation loss exceeds the set increase threshold, the confidence of the boundary sample decreases and the easily confused category shift increases, the parameter block that exceeds the coverage of the generalization transition shell block in the corresponding round is marked as the generalization collapse boundary block. All parameter blocks constitute the generalization collapse boundary block set. Using the same parameter block number as anchor, stable core block, generalized transition shell block and generalized collapse boundary block with the same number are combined into shell domain ternary units. Unidirectional reachable continuation edges are established between shell domain ternary units based on the network layer adjacency relationship, temporal continuity relationship and gradient direction consistency relationship. After eliminating non-monotonic energy density paths, the continuous three-layer shell-like region enclosed by the shell domain ternary units and reachable continuation edges is determined as the reachable generalized manifold shell domain. According to the level of each parameter block in the shell domain, it is mapped to the stable core region, the generalized transition shell region and the generalized collapse boundary region respectively.

4. The method for optimizing a training model based on deep learning according to claim 1, characterized in that, The process of generating shell domain assignment results for each parameter block in the reachable generalized manifold shell domain through dynamic mixing includes: Read the parameter block number, network layer number, parameter norm, parameter update magnitude, gradient direction change, training loss decrease, validation loss change, and parameter energy density corresponding to each parameter block, and generate parameter block label objects according to the field order; Read the stable kernel block, generalized transition shell block and generalized collapse boundary block in the reachable generalized manifold shell domain, generate stable kernel point labeled objects, transition shell labeled objects and collapse boundary labeled objects respectively, and combine all labeled objects into a shell domain labeled sequence according to the training round order. The shell domain labeled sequence is input into the improved HyperMixer model. The improved HyperMixer model consists of a label encoding layer, a position index embedding layer, a layer normalization layer, a hypernetwork weight generation layer, a dynamic token-mixing layer, and an output mapping layer. The label encoding layer performs linear projection on each label object in the shell domain label sequence according to the field order, mapping the vector after concatenating the original fields into a 128-dimensional representation vector; Location index embedding layer: Adds location information from the shell domain label sequence to each representation vector, and adds the location information vector to the representation vector element by element; Layer normalization layer: The representation vector after adding position information is subjected to mean zeroing and variance normalization to obtain a normalized vector; Supernetwork weight generation layer: Receives a normalized vector and combines it with the embedding vector and parameter energy density of the current training stage, and generates dynamic mixed weights through a two-level fully connected mapping. Dynamic token-mixing layer: Four parallel mixing heads are set up. Each mixing head performs channel mixing and label mixing operations on parameter block marker objects, stable kernel point marker objects, transition shell marker objects and collapsed boundary marker objects respectively according to dynamic mixing weights, and outputs the mixed parameter block representation vector. Output mapping layer: Performs fully connected mapping on the mixed parameter block representation vector, and outputs the stable kernel region matching score, the generalized transition shell region matching score, the generalized collapsed boundary region matching score, and the shell domain deviation region matching score. The matching scores of the stable core region, the generalized transition shell region, the generalized collapse boundary region, and the shell domain deviation region corresponding to each parameter block are sorted, and the region corresponding to the maximum matching score is determined as the shell domain assignment result of the parameter block in the reachable generalized manifold shell domain. Output the shell domain attribution results corresponding to each parameter block and write them into the parameter block state set.

5. The method for optimizing a training model based on deep learning according to claim 1, characterized in that, The determination that the current model parameters have a risk of detachment from the reachable generalization manifold shell domain includes: Read the model parameter snapshots of the deep learning model to be optimized in the order of training rounds, and expand the parameter values ​​of the same parameter block in each training round into a parameter block vector according to the network layer number, parameter block number and parameter dimension. For two adjacent training rounds of the same parameter block, perform bitwise interpolation processing, subtract the parameter block vector of the previous training round from the parameter block vector of the later training round, and obtain the parameter increment vector of the parameter block between the corresponding training rounds. Store all parameter increment vectors in the order of training rounds. Connect the parameter increment vectors of the same parameter block according to the training rounds, set the starting point of each parameter increment vector to the parameter position of the previous training round, and set the ending point to the parameter position of the next training round to generate the parameter transition trajectory of the parameter block. The direction change of two adjacent parameter increment vectors in the parameter transition trajectory is calculated. First, the length and direction sign of the two parameter increment vectors are read, then the direction angle between the two parameter increment vectors is calculated, and the direction angle is converted into radians to obtain the radians of the parameter transition trajectory of the parameter block in the corresponding training round. The parameter transition trajectory radii of the same parameter block are continuously compared in the order of training rounds. When the parameter transition trajectory radii of the same parameter block increase sequentially in three consecutive training rounds, and the shell domain assignment result corresponding to the parameter block is a generalization collapse boundary region or an off-shell region, the parameter block is marked as a deviance risk parameter block. Based on the parameter block number, network layer number, training round number, parameter transition trajectory, parameter transition trajectory radii, and shell domain assignment result of the deviance risk parameter block, a deviance risk result for the reachable generalization manifold shell domain is generated.

6. The method for optimizing a training model based on deep learning according to claim 1, characterized in that, The step of projecting model parameters at risk of detachment onto a locally stable spanned subspace and generating reconstructed model parameters based on the shell domain attribution results includes: In response to the detection of detachment risk from the reachable generalized manifold shell domain, the parameter block number, network layer number, training round number, and shell domain assignment result of the detachment risk parameter block are read, and stable kernel point blocks with the same parameter block number as the detachment risk parameter block are extracted from the stable kernel region of the reachable generalized manifold shell domain. The stable kernel blocks are sorted according to the order of verification loss from low to high, parameter energy density from low to high, and training epoch from recent to distant. Stable kernel blocks that meet the preset selection quantity and are not connected to the generalization collapse boundary region are retained to generate a candidate set of stable kernel blocks. The stable kernel block with the lowest verification loss in the candidate set of stable kernel points is used as the local reference block. The remaining stable kernel blocks are subjected to parameter difference processing with the local reference block to generate stable support directions. Stable support directions that point to the generalized collapse boundary region or the deviation region outside the shell domain are eliminated to generate the anti-collapse stable support set. Based on the adjacency relationship of the network layer, the consistency relationship of parameter block numbering, and the continuity relationship of stable support direction, the stable support directions in the anti-collapse stable support set are combined into a local stable support skeleton, and the parameter activity range jointly defined by the local reference block, the candidate set of stable core points, and the local stable support skeleton is determined as the local stable tensor subspace. Model parameters at risk of deviating are projected into a locally stable tensor subspace. Based on the shell domain assignment results, the parameter blocks in the stable core region are updated by maintaining their position, the parameter blocks in the generalized transition shell region are updated by limiting their amplitude, the parameter blocks in the generalized collapse boundary region are updated by projection rollback, and the parameter blocks in the off-topic region outside the shell domain are updated by low-rank reconstruction. The corresponding network layer parameter positions are written back according to the parameter block number to generate the reconstructed model parameters.

7. The method for optimizing a training model based on deep learning according to claim 1, characterized in that, The optimized deep learning training model includes: Read the validation loss, training loss, boundary sample confidence, easily confused sample class offset, and number of out-of-shell deviation parameter blocks corresponding to the model parameters before reconstruction, and generate benchmark validation records; The reconstructed model parameters are written into the model to be validated, and forward validation is performed on the model to be validated using the validation dataset to generate a reconstruction validation record. The reconstruction validation record includes the reconstruction validation loss, the confidence of the reconstruction boundary samples, the class offset of the reconstructed easily confused samples, and the number of out-of-shell deviation parameter blocks. The training loss of the model to be validated is reviewed using the training dataset to generate a reconstructed training loss. The reconstructed training loss is then compared with the training loss in the benchmark validation record to obtain the change in training loss. When the reconstructed validation loss is lower than the validation loss in the benchmark validation record, the confidence of the reconstructed boundary samples is not lower than the confidence of the boundary samples in the benchmark validation record, the class offset of the reconstructed easily confused samples is not greater than the class offset of the easily confused samples in the benchmark validation record, the number of reconstructed out-of-shell deviation parameter blocks is less than the number of out-of-shell deviation parameter blocks in the benchmark validation record, and the change in training loss does not exceed the preset training bounce threshold, the reconstructed model parameters are determined as the optimized model parameters. Based on the optimized model parameters, continue to perform model training. Repeat the parameter block state update, shell domain attribution judgment, detachment risk detection and parameter reconstruction verification in consecutive training rounds. When no detachment risk result of reachable generalized manifold shell domain is generated within a consecutive preset number of rounds, and the verification loss change meets the preset stability condition, output the optimized deep learning training model.

Images