Task semantic perception-based large model self-adaptive training method and system

Abstract

This invention relates to the field of semantic processing technology, specifically to a large-scale adaptive training method and system based on task semantic awareness. The method includes the following steps: acquiring text structure information, extracting batch semantic vectors to determine consistency, tracking semantic differences across steps to form a trajectory, merging stage boundaries, locating abnormal nodes and verifying loss gradients, and mapping parameter tuning instructions into an adaptive training control structure. In this invention, through structured semantic expression and a real-time feedback mechanism, the training process can be dynamically adjusted to adapt to changes in task semantics, avoiding semantic shift and inconsistency issues. Precise labeling of semantic states and monitoring of change trajectories enable the model to adjust training strategies in a timely manner at different semantic stages, thereby improving training efficiency and stability. Accurate location of key nodes allows training to focus on training steps with significant impact, avoiding ineffective training. Through semantic-driven parameter adjustment and training control, the model's adaptability to complex tasks and resource utilization are improved.

Description

Technical Field

[0001] This invention relates to the field of semantic processing technology, and in particular to a method and system for adaptive training of large models based on task semantic awareness. Background Technology

[0002] The field of semantic processing technology encompasses the identification, modeling, and utilization of the meanings carried by words, sentences, and texts in natural language. The core of this technology lies in completing the mapping and constraint between task objectives and semantic information through the formal representation of linguistic symbols and semantic relationships. It typically revolves around semantic unit representation, semantic relationship modeling, task semantic description, and semantic consistency constraints. Overall, it covers technical directions such as semantic representation construction, task semantic modeling, semantic feature constraints, and semantic alignment during the training process, in order to support the language understanding and processing needs under different task conditions.

[0003] Among them, the large model adaptive training method and system based on task semantic awareness refers to the technical solution of introducing explicit task semantic description information during the training process and adjusting the training behavior accordingly. This patent topic addresses the problem of training differences of large models under different task conditions, covering task semantic information acquisition, correspondence between task semantics and sample semantics, semantic annotation of training samples and task targets, and adjustment of training parameters as task semantics change. Specifically, it uses the structured expression of the semantic elements of the input task as the basis for training constraints, participates in sample selection, gradient calculation, path control, and parameter update rule setting, thereby completing the adaptive training process associated with task semantics.

[0004] Existing technologies primarily utilize task semantics during training by focusing on static annotations at the pre-training or sample level. The semantic state lacks continuous characterization and feedback channels during training, making it difficult to perceive semantic changes in a timely manner. Training behavior typically relies on fixed loss optimization paths, lacking monitoring mechanisms for the evolution of intermediate semantic representations. When semantic distribution undergoes a phase shift, the training strategy remains unchanged, easily leading to the accumulation of semantic biases. Existing solutions have coarse-grained control over the training process, lacking clear boundaries between training stages. This results in parameter updates being performed in the same way in semantically stable and semantically changing regions, hindering training efficiency. Abnormal training steps or key turning points are often masked by the overall optimization process, making it difficult to identify and correct the impact of local inconsistencies on the final result. In multi-task or complex semantic scenarios, this operating mode easily leads to training process oscillations, decreased convergence speed, and increased resource consumption, constraining the model's performance under specific task conditions. Summary of the Invention

[0005] To address the technical problems existing in the prior art, embodiments of the present invention provide a large-scale adaptive training method for models based on task semantic awareness, comprising the following steps: To achieve the above objectives, the present invention adopts the following technical solution: a large-model adaptive training method based on task semantic awareness, comprising the following steps: S1: Obtain the input text and structured information of the training batch, perform semantic embedding, form a set of semantic feature vectors, calculate the distance between vectors for aggregation judgment, classify and label the semantic consistency status of the batch based on the results, and generate a semantic state structure. S2: Based on the semantic state structure, collect the semantic representation of the intermediate layer of the model in consecutive training steps, calculate the semantic difference between adjacent steps and arrange them in chronological order to generate a semantic change trajectory; S3: Call the semantic change trajectory, judge and identify the consistency of adjacent semantic states, merge continuous semantic changes into the same stage, mark the break or turn as the stage boundary, and generate the stage division structure. S4: Based on the stage division structure, within each semantic stage, compare the local changes of the evolution trajectory, filter out steps that are inconsistent with the overall trend, and perform cross-validation by combining loss and gradient changes to generate a key node localization structure. S5: Based on the semantic state structure, the stage division structure and the key node positioning structure, map each stage and key node to the training parameter adjustment instructions to generate an adaptive training control structure.

[0006] As a further aspect of the present invention, the semantic state structure includes a batch semantic consistency marker, a set of semantic feature vectors, and an aggregated judgment result; the semantic change trajectory includes the amount of semantic change, temporal order arrangement, and step difference sequence. The phase division structure includes phase markers, phase boundaries, and semantic state consistency evaluation. The key node localization structure includes local evolution abnormal nodes, gradient change key nodes, and loss abnormal steps; The adaptive training control structure includes a training phase adjustment scheme, key node adjustment instructions, and phase progress control strategy.

[0007] As a further aspect of the present invention, the mutation identification of adjacent semantic states refers to comparing the magnitude and direction of adjacent semantic changes in consecutive training steps; The stage boundary refers to the position where, during the semantic change process, there is an interruption or a change in direction based on the trend of change.

[0008] As a further aspect of the present invention, the evolution trajectory refers to the time-series process description formed by continuously tracking the changes in the intermediate semantic representation of the model as the training steps progress; Cross-validation refers to the process of simultaneously comparing and analyzing the changes in loss and gradient when semantic changes deviate, in order to determine the degree of impact of that step on the training process.

[0009] As a further aspect of the present invention, the specific steps of S1 are as follows: S101: Obtain the input text and structured information of the training batch, call the mapping relationship between the text tag sequence and the field key value, extract semantic units according to the character sequence order and field position index, perform vector mapping operation on the semantic units and unify the dimension scale, and generate a set of semantic feature vectors. S102: Based on the set of semantic feature vectors, calculate the distance between vectors and construct a distance matrix. Based on the distance matrix, statistically analyze the distance distribution intervals between samples. Based on the proportion of the number of samples corresponding to each distribution interval, determine the aggregation state and map the aggregation state to a classification label to generate a semantic consistency classification result. S103: Based on the semantic consistency classification results, organize the batch sample states according to the classification label order, call the relationship between classification labels and batch indexes, perform state mapping and structural arrangement for different consistency intervals, and establish a semantic state structure.

[0010] As a further aspect of the present invention, the specific steps of S2 are as follows: S201: Based on the semantic state structure, collect the semantic representation of the intermediate layer of the model in consecutive training steps, obtain the semantic vector of adjacent training steps, perform difference calculation element by element, arrange the difference sequence according to the step order, and obtain the semantic step difference value sequence. S202: Based on the semantic step difference value sequence, obtain the time index of adjacent steps, calculate the cumulative offset of the difference amplitude under the time index, compare the offset amplitude of each time point and integrate the time density term and the hierarchical distribution term to obtain the semantic difference intensity value and generate a time-ordered difference curve. S203: Call the time-ordered difference curve, splice the curve values ​​according to the time index, determine the direction and magnitude of change in adjacent time periods, connect the change segments, and establish a semantic change trajectory.

[0011] As a further aspect of the present invention, the specific steps of S3 are as follows: S301: Based on the semantic change trajectory, determine the consistency of adjacent semantic states, obtain the time period of continuous semantic change, select a threshold for filtering based on the semantic change amplitude, and iteratively filter by combining the ratio of amplitude to time period length to obtain the semantic state change time period. S302: Based on the semantic state change time period, determine the breakpoints and turning points in the semantic trajectory, perform threshold comparison analysis based on the change amplitude and duration, calculate the change rate and filter the boundary points that meet the conditions, and identify and mark the stage boundary markers by comparing the amplitude and duration. S303: Invoke the stage boundary marker, merge adjacent semantic changes, and merge continuous semantic changes into the same stage according to the time period division rules to generate a stage division structure.

[0012] As a further aspect of the present invention, the specific steps of S4 are as follows: S401: Based on the stage division structure, within each semantic stage, obtain the evolution trajectory data of each step, calculate the change direction of each step, compare it with the overall trend direction, filter out steps that are inconsistent with the overall trend based on the direction consistency benchmark value, and generate a set of trend deviation steps. S402: Based on the set of trend deviation steps, obtain the loss change data and gradient change data of the deviation steps, calculate the loss difference and gradient difference of each deviation step, compare the loss and gradient difference to obtain the node consistency value, judge the consistency value and the judgment benchmark value, filter the candidate key node values ​​that meet the conditions, and generate a candidate key node set. S403: Call the candidate key node set, sort the node consistency values, and filter according to the sorting results and node stability judgment interval to generate the key node positioning structure.

[0013] As a further aspect of the present invention, the specific steps of S5 are as follows: S501: Based on the semantic state structure and stage division structure, obtain the evolution data of each stage, analyze and compare the stage evolution characteristics with the node characteristics in the key node positioning structure, calculate the adaptive differences, filter the node data that meets the threshold, and obtain the mapping data set. S502: Based on the mapping data set, process the evolution data of each stage, monitor the mapping relationship with the training parameter adjustment instructions, calculate the parameter change and evolution feature change of each node, obtain the node fitness value, compare it with the fitness benchmark value, screen the adaptive training control nodes that meet the conditions, and establish an adaptive training control structure. S503: Invoke the adaptive training control structure, sort the adaptive values ​​of each node, filter them according to the sorting results and the adaptive control standard range, and generate an adaptive training control instruction structure.

[0014] A task-semantic-aware large-model adaptive training system includes: The semantic input processing module acquires the input text and structured information of the training batch, performs semantic embedding, generates a set of semantic feature vectors, calculates the Euclidean distance between vectors, clusters the vectors based on the results, classifies and labels the semantic consistency state of the batch, and establishes a semantic state structure. The semantic state classification module collects the semantic representations of the intermediate layers of the model in consecutive training steps based on the semantic state structure, calculates the semantic changes in adjacent steps, arranges them in chronological order, and obtains the semantic change trajectory. The semantic trajectory analysis module calls the semantic change trajectory, judges the consistency of adjacent semantic states, identifies abrupt changes and merges continuous changes into the same stage, marks the stage boundaries, and forms a stage division structure. Based on the stage division structure, the stage division module compares local changes within each semantic stage, filters out steps that are inconsistent with the overall trend, and performs cross-validation by combining loss and gradient changes to establish a key node localization structure. The training adjustment module maps each stage and key node to training parameter adjustment instructions based on the semantic state structure, the stage division structure, and the key node positioning structure, thereby generating an adaptive training control structure.

[0015] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, through structured semantic expression and real-time feedback mechanisms, the training process can be dynamically adjusted to adapt to changes in task semantics, avoiding semantic deviation and inconsistency issues. The accurate labeling of semantic states and monitoring of change trajectories enable the model to adjust training strategies in a timely manner at different semantic stages, thereby improving training efficiency and stability. The precise location of key nodes allows training to focus on training steps with significant impact, avoiding ineffective training. Through semantic-driven parameter adjustment and training control, the model's adaptability to complex tasks and resource utilization are improved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the steps of the present invention; Figure 2 This is a detailed schematic diagram of S1 of the present invention; Figure 3 This is a detailed schematic diagram of S2 of the present invention; Figure 4 This is a detailed schematic diagram of S3 of the present invention; Figure 5 This is a detailed schematic diagram of S4 of the present invention; Figure 6 This is a detailed schematic diagram of S5 of the present invention; Figure 7 This is a system module diagram of the present invention. Detailed Implementation

[0018] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0019] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0020] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0021] In embodiments of the present invention, sometimes the subscript is as follows: It may be written in a non-subscript form such as W1, and when the distinction is not emphasized, the meaning they express is the same.

[0022] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0023] Please see Figure 1 This invention provides a large-scale adaptive training method for models based on task semantic awareness, comprising the following steps: S1: Obtain the input text and structured information of the training batch, perform semantic embedding, form a set of semantic feature vectors, calculate the distance between vectors for aggregation judgment, classify and label the semantic consistency status of the batch based on the results, and generate a semantic state structure. S2: Based on the semantic state structure, collect the semantic representation of the intermediate layer of the model in consecutive training steps, calculate the semantic difference between adjacent steps and arrange them in time order to generate a semantic change trajectory; S3: Call the semantic change trajectory, judge and identify the consistency of adjacent semantic states through mutation, merge continuous semantic changes into the same stage, mark the break or turn as the stage boundary, and generate the stage division structure. S4: Based on the stage division structure, within each semantic stage, compare the local changes of the evolution trajectory, filter out steps that are inconsistent with the overall trend, and perform cross-validation by combining loss and gradient changes to generate the key node localization structure. S5: Based on semantic state structure, stage division structure and key node localization structure, each stage and key node are mapped to training parameter adjustment instructions to generate an adaptive training control structure. The semantic state structure includes a batch semantic consistency marker, a set of semantic feature vectors, and an aggregated judgment result; Semantic change trajectories include the amount of semantic change, temporal order, and step-by-step difference sequence. The phase division structure includes phase markers, phase boundaries, and semantic state consistency assessment; The key node localization structure includes local evolution anomaly nodes, gradient change key nodes, and loss anomaly steps; The adaptive training control structure includes training phase adjustment schemes, key node adjustment instructions, and phase progress control strategies.

[0024] Please see Figure 2 The specific steps of S1 are as follows: S101: Obtain the input text and structured information of the training batch, call the mapping relationship between the text tag sequence and the field key value, extract semantic units according to the character sequence order and field position index, perform vector mapping operation on the semantic units and unify the dimension scale, and generate a set of semantic feature vectors. In training batch processing, the input text and its corresponding structured field information are first obtained one by one from the same batch. The structured field information includes at least the field identifier and the character position index range. For example, if the text length of a sample is 30 characters, the field... Corresponding character position , field Corresponding character position , field Corresponding character position Then, based on the character sequence order and field position index, the text is segmented, extracting consecutive characters within the same field index range into a semantic unit, such as the field... Extracting semantic units consisting of 4 characters, the field A semantic unit consisting of 9 characters is extracted. Vector mapping is performed on each semantic unit, mapping each character to a numeric vector of length 128. Then, the character vectors within the same semantic unit are summed dimension-wise and divided by the number of characters. For example, in the field... The four character vectors are summed dimension by dimension and then divided by 4 to obtain a 128-dimensional semantic vector. The dimensions of the semantic vectors obtained from different fields are checked; if the dimension is less than 128, zeros are padded to the end of the vector to bring it to 128. Then, the maximum and minimum values ​​of all semantic vector elements in the current batch are calculated. Assuming the maximum value is 2.5 and the minimum value is -1.0, the minimum value is subtracted from each element and then divided by the range 3.5 to ensure that all vector elements fall within the range. The intervals are then processed, and the semantic vectors are stored in a set structure according to the sample index and field order to form a set of semantic feature vectors. S102: Based on the set of semantic feature vectors, calculate the distance between vectors and construct a distance matrix. Based on the distance matrix, statistically analyze the distance distribution intervals between samples. Determine the aggregation state based on the proportion of samples corresponding to each distribution interval, and map the aggregation state to a classification label to generate a semantically consistent classification result. Based on the aforementioned set of semantic feature vectors, semantic consistency is determined for samples within the same training batch. Specifically, any two samples in the batch are compared in pairs. For example, if a batch contains three samples, three sample pairs are formed. For each sample pair, the corresponding semantic vectors are first aligned according to the field order, and then the vector distance is calculated field by field. For the semantic vector of each field, the difference is calculated dimension by dimension, the difference is squared, summed, and then the square root is taken. For example, if the sum of squares for a certain field is 4.0, then the distance for that field is 2.0. Subsequently, the distance values ​​of all fields in the same sample pair are summed and divided by the number of fields, 3, to obtain the average distance value of the sample pair, such as (2.0 + 1.5 + 2.5) / 3 = 2.0. The average distance values ​​of all sample pairs are filled into the distance matrix, and the off-diagonal elements in the distance matrix are statistically analyzed to divide the distance interval into... The distance count within each interval is then calculated. If all three distance values ​​fall into one of the three intervals, the percentage for each interval is 33.3%. This percentage is then compared to a preset threshold. An interval with a percentage greater than or equal to 50% is marked as a concentrated interval; an interval between 20% and 50% is marked as a transitional interval; and an interval less than or equal to 20% is marked as a discrete interval. Since the percentage in this example is 33.3%, the corresponding interval is marked as a transitional interval, and this interval's state is mapped to a semantic consistency classification label. Generate semantically consistent classification results; S103: Based on the semantic consistency classification results, organize the batch sample states according to the classification label order, call the relationship between classification labels and batch indexes, perform state mapping and structural arrangement for different consistency intervals, and establish a semantic state structure; Based on the semantic consistency classification results, semantic consistency classification labels are obtained. State structure construction is then performed on the training batch samples, specifically by reading the classification label corresponding to each sample. When samples 1, 2, and 3 all correspond to a label... At that time, the sample index is sorted according to the preset priority order of classification labels. If the labels are the same, the original index order is maintained. Then, according to the mapping relationship between classification labels and status codes, the labels are... Convert to status code The data is written into the state field of the corresponding sample. Then, sample indices with the same state code are combined into a state structure unit. The sample index set and the corresponding distance statistics are recorded in the structure unit. The distance statistics directly adopt the average distance result of 2.0 obtained above and store this value as the numerical attribute of the structure unit. Finally, all state structure units are arranged in batch order to form a semantic state structure for representing the semantic consistency state of the training batch.

[0025] Please see Figure 3 The specific steps of S2 are as follows: S201: Based on the semantic state structure, the semantic representation of the intermediate layer of the model in consecutive training steps is collected, the semantic vector of adjacent training steps is obtained, the difference is calculated element by element, and the difference sequence is arranged in the order of the steps to obtain the semantic step difference sequence. Based on the aforementioned semantic state structure, the semantic representations of the intermediate layers of the model are collected in consecutive training steps. Each training step corresponds to a semantic vector of length 128. Let the time indices of the consecutive training steps be... Between adjacent steps, element-wise difference operations are performed on each dimension of the same semantic vector. The difference method is to subtract the element corresponding to the previous time index from the element corresponding to the later time index. For example, in sample 1, the first dimension element is... and The values ​​for the sub-divisions are 0.42 and 0.45, respectively. Therefore, the difference result for this dimension is 0.45 - 0.42 = 0.03. Following the same rule, the difference vectors are calculated for each of the 128,128,128 dimensions, and then... This difference operation is performed sequentially, and finally all difference vectors are arranged in time index order to form a semantic step difference value sequence; S202: Based on the semantic step difference value sequence, obtain the time index of adjacent steps, calculate the cumulative offset of the difference amplitude under the time index, compare the offset amplitude of each time point and integrate the time density term and the hierarchical distribution term to obtain the semantic difference intensity value and generate a time-ordered difference curve. After obtaining the semantic step difference sequence, a time index interval corresponding one-to-one with each difference vector is introduced. And for each difference vector, the difference magnitude is calculated. The difference magnitude is obtained by summing the absolute values ​​of the elements in each dimension of the difference vector and dividing by the number of dimensions. For example, in In the example, the sum of the absolute values ​​of the 128-dimensional differences for sample 1 is 3.203, so its difference magnitude is 3.20 / 128 = 0.025. Then, the average difference magnitudes of all samples within the same time interval are taken to obtain the base offset under that time index. The base offset is 0.027. Subsequently, a cumulative operation is performed on the base offset in chronological order to form a cumulative offset sequence. The cumulative offset is updated to 0.027 + 0.031 = 0.058, and a time density term and a hierarchical distribution term are introduced based on this. The time density term is obtained by multiplying the number of samples by the number of fields, and its value is... The hierarchical distribution item is determined by the amplitude ratio of different fields. The cumulative offset, time density item and hierarchical distribution item are multiplied and summed in turn to obtain the semantic difference intensity value corresponding to each time index, and a time-ordered difference curve is formed in time order. S203: Call the time-ordered difference curve, stitch the curve values ​​according to the time index, determine the direction and magnitude of change in adjacent time periods, connect the change segments, and establish a semantic change trajectory. Based on the temporally ordered difference curve, semantic difference intensity values ​​are read in time index order to form an intensity sequence (0.243, 0.522, 0.702, 0.864). Difference operations are performed on the intensity values ​​at adjacent time points to determine the direction of change. For example, in... The calculation 0.522 - 0.243 = 0.279 shows a positive result, confirming an upward trend. Simultaneously, the change range is compared with a preset range. The values ​​greater than 0.3 are compared and judged. The above changes fall into the corresponding intervals. Then, a connection operation is performed on adjacent time periods with the same direction of change and continuous amplitude intervals. Connect the segments to the same change segment and record their start and end time indices and corresponding intensity values ​​of 0.243 and 0.864. Finally, connect the change segments in chronological order to construct the semantic change trajectory.

[0026] Please see Figure 4 The specific steps of S3 are as follows: S301: Based on the semantic change trajectory, determine the consistency of adjacent semantic states, obtain the time period of continuous semantic change, select a threshold for filtering based on the semantic change amplitude, and iteratively filter by combining the ratio of amplitude to time period length to obtain the semantic state change time period. Based on the obtained semantic change trajectory, the continuous time index Using the corresponding semantic difference intensity values ​​(0.243, 0.522, 0.702, 0.864) as input, a consistency judgment is performed on adjacent time points. First, the difference magnitude between adjacent time points is calculated, and then... Taking the absolute value of the above difference yields the amplitude sequence (0.279, 0.180, 0.162), where all amplitude values ​​are within the normalized range. Within the interval, an amplitude threshold is then calculated based on the amplitude sequence. By averaging the amplitudes And round to two decimal places to get Compare each amplitude value with Comparison, only The filter criteria are met, and the corresponding time period is: Then, a time period length parameter is introduced, denoted as . Calculate the ratio of amplitude to length. and adopt with Consistent ratio threshold Perform a second screening to meet the requirements. Based on this, it is determined The semantic state change time period is used as the output result of this step.

[0027] S302: Based on the time period of semantic state change, determine the breakpoints and turning points in the semantic trajectory, perform threshold comparison analysis based on the change amplitude and duration, calculate the change rate and filter the boundary points that meet the conditions, and identify and mark the stage boundary markers by comparing the amplitude and duration. As an embodiment supporting the claim of "determining the breakpoints and turning points in the semantic trajectory and marking the stage boundaries", this is based on the aforementioned differential amplitude sequence (0.279, 0.180, 0.162) and its corresponding boundary time points. First, the breakpoint is determined, and the minimum amplitude is taken. The fracture threshold is obtained by rounding down to one decimal place. Compare each amplitude value with The comparison shows that neither of them satisfies the requirements. Therefore, no fracture point is generated. Next, the turning point judgment is performed. By comparing the signs of the differences between adjacent strength values, all three differences are positive, indicating no direction reversal. The set of turning points is empty. Subsequently, stage boundary filtering is performed, introducing the rate of change parameter. When the length of each segment is 1, we get Then, the boundary threshold is calculated based on the maximum amplitude value. , and the rates of change Comparison, only It is established, and the corresponding time boundary point is The comparison of amplitude and duration confirmed that its change was most significant under the same duration, ultimately leading to Mark them as stage boundary points and output them to the stage boundary marker set; S303: Invoke the stage boundary marker, merge adjacent semantic changes, and merge continuous semantic changes into the same stage according to the time period division rules to generate a stage division structure; As an embodiment supporting the claim of "generating a stage partitioning structure based on stage boundary markers", a set of stage boundary markers is invoked. The time index sequence is segmented, dividing the overall timeline into two candidate segments. and Then, stage parameters are calculated for each candidate segment. Calculate the net change Stage length average rate of change ,exist Calculate the net change Stage length average rate of change and using the same threshold as mentioned above. Conduct phased confirmation and judgment. and The process consists of two independent stages. In the second stage, adjacent semantic changes are directly merged according to their temporal continuity. Finally, the stage start and end time index, net change amount, and average change rate are written into the stage division structure to form a stage division result that corresponds one-to-one with the semantic change trajectory.

[0028] Please see Figure 5 The specific steps of S4 are as follows: S401: Based on the stage division structure, within each semantic stage, the evolution trajectory data of each step is obtained, the change direction of each step is calculated, and it is compared with the overall trend direction. Based on the direction consistency benchmark value, steps that are inconsistent with the overall trend are filtered out, and a set of trend deviation steps is generated. In a phased structure, phase ,stage It has been determined that the semantic difference intensity values ​​within each stage will be read separately, where The inner range is (0.243, 0.522). The inner values ​​are (0.522, 0.702, 0.864). The difference values ​​are calculated for adjacent steps to obtain... All three values ​​are positive, indicating a positive overall trend for the stage. Subsequently, the field changes at each differencing step are read at the field level. Middle field The changes were +0.120, -0.010, and +0.070, respectively. And satisfying that the sum of the changes in the fields equals 0.180 and 0.162 respectively, for each difference step, count the number of fields whose field direction has the same sign as the overall trend direction of the stage, and obtain the direction consistency ratio. Calculate the directional consistency benchmark value ; S402: Based on the set of trend deviation steps, obtain the loss change data and gradient change data of the deviation steps, calculate the loss difference and gradient difference for each deviation step, compare the loss and gradient differences, obtain the node consistency value, judge the consistency value and the judgment benchmark value, filter the candidate key node values ​​that meet the conditions, and generate a candidate key node set. For trend deviation step set Read the loss value and gradient norm value at the corresponding time index, where Calculate the difference in loss Calculate the gradient difference Based on this, the node consistency value is calculated. Calculate the consistency judgment benchmark value ,implement Judgment, only If the conditions are met, index the corresponding termination time. Write to the candidate key node set ; S403: Call the candidate key node set, sort the node consistency values, and filter based on the sorting results and node stability judgment intervals to generate the key node location structure. Call the candidate key node set The node sequence is obtained by sorting the nodes according to their consistency values. Then, stability interval determination is performed on the candidate nodes, and the gradient difference corresponding to the node is read. With intensity amplitude The stability upper bound is obtained by calculating the median value from the set of gradient differences of deviation steps {0.35, 0.10} and rounding it. The judgment is true, and at the same time... Compared with the aforementioned amplitude benchmark value After comparing and determining that it falls within the low amplitude range, the node time index is finally determined. The consistency value (0.166667), loss difference (0.02), gradient difference (0.10), intensity amplitude (0.162), and direction consistency ratio (0.667) are written together into the key node positioning structure to complete the output of the key node positioning results.

[0029] Please see Figure 6 The specific steps of S5 are as follows: S501: Based on the semantic state structure and stage division structure, obtain the evolution data of each stage, analyze and compare the stage evolution characteristics with the node characteristics in the key node localization structure, calculate the adaptive differences, filter the node data that meets the threshold, and obtain the mapping data set. Based on the semantic state structure, the consistency marker is C2, and the state encoding is... Based on the phase division results, the training process is divided into the first phase (time interval). to ) and the second phase (time interval) to The semantic difference intensity values ​​at the start and end times of each stage were read. In the first stage, the net change was calculated to be 0.279, and the average change rate was 0.279. In the second stage, the net change was calculated to be 0.342, and the average change rate was 0.171. The key node location structure was invoked to read the key nodes. The intensity amplitude is 0.162, the direction consistency ratio is 0.667, and the stage boundary node is selected. As a control node, its intensity amplitude is 0.279 and its directional consistency ratio is 1.000. Each node is then linked to its respective stage, and the stage change rate and the difference between the node's intensity amplitude and its directional consistency ratio and the baseline value of 0.778 are calculated. A weighted average is then used to obtain the overall fitness difference value. The calculation result is as follows: The node corresponds to 0.005842. The node corresponds to 0.011684. The average of the differences yields a filtering threshold of 0.008763. After filtering, only [the node with the highest value] is retained. The nodes are used to form a mapping dataset. Then, the fitness variance values ​​of all nodes are averaged to obtain a threshold of 0.008763. Nodes with fitness variance values ​​less than or equal to this threshold are filtered out, and finally only node B1 is retained, forming the mapping dataset. This dataset contains time indices. The nodes and their corresponding fitness difference values; S502: Based on the mapping dataset, process the evolution data of each stage, monitor the mapping relationship with the training parameter adjustment instructions, calculate the parameter change and evolution feature change of each node, obtain the node fitness value, compare it with the fitness benchmark value, screen the adaptive training control nodes that meet the conditions, and establish the adaptive training control structure. Based on the mapped data set, for nodes The parameter adjustment records for the corresponding training steps are read, and the learning rate values ​​for the two steps before and after the node are obtained as 0.0012 and 0.0010, respectively, with a momentum value of 0.90 for both. The absolute difference is used to calculate the change in learning rate as 0.0002 and the change in momentum as 0. The sum of the two yields the change in node parameters as 0.0002. Simultaneously, the evolutionary feature change for the corresponding stage is read, and the intensity amplitude value of stage one (0.279) is taken. The node parameter change is divided by the evolutionary feature change to calculate the node fitness value as 0.000717. To construct the fitness baseline, non-mapping nodes in the same batch are... Following the same calculation process, the parameter change of this node is read as 0.0002, and the evolutionary characteristic change is 0.162. Its fitness value is calculated to be 0.001235. The fitness values ​​of the two nodes are then averaged to obtain a baseline fitness value of 0.000976. The fitness values ​​of each node are compared with the baseline fitness value, and only nodes with fitness values ​​greater than or equal to the baseline value are retained. The filtered results are then used as a time index. The node is written into the adaptive training control structure, and its stage is recorded as the second stage. The average stage change rate is 0.171, and the node consistency value is 0.166667. S503: Call the adaptive training control structure, sort the adaptive values ​​of each node, filter them according to the sorting results and the adaptive control standard range, and generate an adaptive training control instruction structure. The adaptive training control structure is invoked, and the nodes it contains are sorted according to their fitness values. Since the structure only contains time indices... For a single node, its sorting result remains unchanged. Then, an adaptive control standard interval is constructed, with the lower limit set at the adaptive baseline value of 0.000976 and the upper limit set at the three significant digits of the current node's adaptive value of 0.00124. After determining that the node's adaptive value of 0.001235 is within the above interval, the parameter adjustment instruction for the corresponding training step is read. The learning rate is adjusted from 0.0008 to 0.0006, while the momentum remains unchanged at 0.90. The change in the learning rate instruction is calculated to be -0.0002, and the change in the momentum instruction is 0. Finally, the node time index, node adaptive value, change in learning rate, change in momentum, change in the node's corresponding evolutionary feature of 0.162, average change rate of stage of 0.171, node consistency value of 0.166667, and direction consistency ratio of 0.667 are integrated to generate an adaptive training control instruction structure.

[0030] Please see Figure 7 A large-scale adaptive training system for task-semantic-aware models, including: The semantic input processing module acquires the input text and structured information of the training batch, performs semantic embedding, generates a set of semantic feature vectors, calculates the Euclidean distance between vectors, clusters the vectors based on the results, classifies and labels the semantic consistency state of the batch, and establishes a semantic state structure. The semantic state classification module collects the semantic representations of the intermediate layers of the model in consecutive training steps based on the semantic state structure, calculates the semantic changes in adjacent steps, arranges them in chronological order, and obtains the semantic change trajectory. The semantic trajectory analysis module calls the semantic change trajectory, judges the consistency of adjacent semantic states, identifies abrupt changes and merges continuous changes into the same stage, marks the stage boundaries, and forms a stage division structure. The phase division module is based on the phase division structure. Within each semantic phase, it compares local changes, filters out steps that are inconsistent with the overall trend, and performs cross-validation by combining loss and gradient changes to establish a key node localization structure. The training adjustment module maps each stage and key node to training parameter adjustment instructions based on the semantic state structure, stage division structure, and key node positioning structure, thereby generating an adaptive training control structure.

[0031] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large-scale adaptive training method for task-semantic-aware models, characterized in that, Includes the following steps: S1: Obtain the input text and structured information of the training batch, perform semantic embedding, form a set of semantic feature vectors, calculate the distance between vectors for aggregation judgment, classify and label the semantic consistency status of the batch based on the results, and generate a semantic state structure. S2: Based on the semantic state structure, collect the semantic representation of the intermediate layer of the model in consecutive training steps, calculate the semantic difference between adjacent steps and arrange them in chronological order to generate a semantic change trajectory; S3: Call the semantic change trajectory, judge and identify the consistency of adjacent semantic states, merge continuous semantic changes into the same stage, mark the break or turn as the stage boundary, and generate the stage division structure. S4: Based on the stage division structure, within each semantic stage, compare the local changes of the evolution trajectory, filter out steps that are inconsistent with the overall trend, and perform cross-validation by combining loss and gradient changes to generate a key node localization structure. S5: Based on the semantic state structure, the stage division structure and the key node positioning structure, map each stage and key node to the training parameter adjustment instructions to generate an adaptive training control structure.

2. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The semantic state structure includes batch semantic consistency markers, a set of semantic feature vectors, and aggregated judgment results; the semantic change trajectory includes semantic change amount, temporal order arrangement, and step difference sequence; the stage division structure includes stage markers, stage boundaries, and semantic state consistency evaluation; the key node localization structure includes local evolution abnormal nodes, gradient change key nodes, and loss abnormal steps; the adaptive training control structure includes training stage adjustment schemes, key node adjustment instructions, and stage progress control strategies.

3. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The mutation identification of adjacent semantic states refers to comparing the magnitude and direction of adjacent semantic changes in consecutive training steps; The stage boundary refers to the position where, during the semantic change process, there is an interruption or a change in direction based on the trend of change.

4. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The evolution trajectory refers to the time-series process description formed by continuously tracking the changes in the intermediate semantic representation of the model as the training steps progress; Cross-validation refers to the process of simultaneously comparing and analyzing the changes in loss and gradient when semantic changes deviate, in order to determine the degree of impact of that step on the training process.

5. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The specific steps of S1 are as follows: S101: Obtain the input text and structured information of the training batch, call the mapping relationship between the text tag sequence and the field key value, extract semantic units according to the character sequence order and field position index, perform vector mapping operation on the semantic units and unify the dimension scale, and generate a set of semantic feature vectors. S102: Based on the set of semantic feature vectors, calculate the distance between vectors and construct a distance matrix. Based on the distance matrix, statistically analyze the distance distribution intervals between samples. Based on the proportion of the number of samples corresponding to each distribution interval, determine the aggregation state and map the aggregation state to a classification label to generate a semantic consistency classification result. S103: Based on the semantic consistency classification results, organize the batch sample states according to the classification label order, call the relationship between classification labels and batch indexes, perform state mapping and structural arrangement for different consistency intervals, and establish a semantic state structure.

6. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The specific steps of S2 are as follows: S201: Based on the semantic state structure, collect the semantic representation of the intermediate layer of the model in consecutive training steps, obtain the semantic vector of adjacent training steps, perform difference calculation element by element, arrange the difference sequence according to the step order, and obtain the semantic step difference value sequence. S202: Based on the semantic step difference value sequence, obtain the time index of adjacent steps, calculate the cumulative offset of the difference amplitude under the time index, compare the offset amplitude of each time point and integrate the time density term and the hierarchical distribution term to obtain the semantic difference intensity value and generate a time-ordered difference curve. S203: Call the time-ordered difference curve, splice the curve values ​​according to the time index, determine the direction and magnitude of change in adjacent time periods, connect the change segments, and establish a semantic change trajectory.

7. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The specific steps for S3 are as follows: S301: Based on the semantic change trajectory, determine the consistency of adjacent semantic states, obtain the time period of continuous semantic change, select a threshold for filtering based on the semantic change amplitude, and iteratively filter by combining the ratio of amplitude to time period length to obtain the semantic state change time period. S302: Based on the semantic state change time period, determine the breakpoints and turning points in the semantic trajectory, perform threshold comparison analysis based on the change amplitude and duration, calculate the change rate and filter the boundary points that meet the conditions, and identify and mark the stage boundary markers by comparing the amplitude and duration. S303: Invoke the stage boundary marker, merge adjacent semantic changes, and merge continuous semantic changes into the same stage according to the time period division rules to generate a stage division structure.

8. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The specific steps of S4 are as follows: S401: Based on the stage division structure, within each semantic stage, obtain the evolution trajectory data of each step, calculate the change direction of each step, compare it with the overall trend direction, filter out steps that are inconsistent with the overall trend based on the direction consistency benchmark value, and generate a set of trend deviation steps. S402: Based on the set of trend deviation steps, obtain the loss change data and gradient change data of the deviation steps, calculate the loss difference and gradient difference of each deviation step, compare the loss and gradient difference to obtain the node consistency value, judge the consistency value and the judgment benchmark value, filter the candidate key node values ​​that meet the conditions, and generate a candidate key node set. S403: Call the candidate key node set, sort the node consistency values, and filter according to the sorting results and node stability judgment interval to generate the key node positioning structure.

9. The large model adaptive training method based on task semantic awareness according to claim 1, characterized in that, The specific steps of S5 are as follows: S501: Based on the semantic state structure and stage division structure, obtain the evolution data of each stage, analyze and compare the stage evolution characteristics with the node characteristics in the key node positioning structure, calculate the adaptive differences, filter the node data that meets the threshold, and obtain the mapping data set. S502: Based on the mapping data set, process the evolution data of each stage, monitor the mapping relationship with the training parameter adjustment instructions, calculate the parameter change and evolution feature change of each node, obtain the node fitness value, compare it with the fitness benchmark value, screen the adaptive training control nodes that meet the conditions, and establish an adaptive training control structure. S503: Invoke the adaptive training control structure, sort the adaptive values ​​of each node, filter them according to the sorting results and the adaptive control standard range, and generate an adaptive training control instruction structure.

10. A large-scale adaptive training system for task semantic awareness, characterized in that, The large model adaptive training method based on task semantic awareness according to any one of claims 1-9, the system comprising: The semantic input processing module acquires the input text and structured information of the training batch, performs semantic embedding, generates a set of semantic feature vectors, calculates the Euclidean distance between vectors, clusters the vectors based on the results, classifies and labels the semantic consistency state of the batch, and establishes a semantic state structure. The semantic state classification module collects the semantic representations of the intermediate layers of the model in consecutive training steps based on the semantic state structure, calculates the semantic changes in adjacent steps, arranges them in chronological order, and obtains the semantic change trajectory. The semantic trajectory analysis module calls the semantic change trajectory, judges the consistency of adjacent semantic states, identifies abrupt changes and merges continuous changes into the same stage, marks the stage boundaries, and forms a stage division structure. Based on the stage division structure, the stage division module compares local changes within each semantic stage, filters out steps that are inconsistent with the overall trend, and performs cross-validation by combining loss and gradient changes to establish a key node localization structure. The training adjustment module maps each stage and key node to training parameter adjustment instructions based on the semantic state structure, the stage division structure, and the key node positioning structure, thereby generating an adaptive training control structure.

Images