Task execution method and apparatus, electronic device, and storage medium

Abstract

This disclosure provides a task execution method, relating to the field of artificial intelligence technology, particularly to the fields of large language models, distributed computing, and deep learning. The specific implementation involves: using at least one hardware device to execute N inference tasks of a large model based on a sequence of N input data units, obtaining N attention distribution information sets and N output data unit sequences. The attention information for the input data units includes at least one candidate attention weight, which is the attention weight between the first data representation of the input data unit in the processing block and the second data representation of the output data unit in the processing block. In response to determining that a first attention distribution information exists among multiple attention distribution information sets, at least one target invalid data unit is determined based on at least one first input data unit corresponding one-to-one with at least one first attention distribution information set. This disclosure also provides a task execution device, an electronic device, and a storage medium.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of generative large models, large language models (LLM), distributed computing, and deep learning, and can be applied to scenarios such as text-to-image generation and question answering. More specifically, this disclosure provides a task execution method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, the application of large language models is constantly increasing. Large language models can not only generate output text based on input text, but also generate content of various modalities based on input multimodal data. Summary of the Invention

[0003] This disclosure provides a task execution method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, a task execution method is provided, the method comprising: using at least one hardware device to execute N inference tasks of a large model based on N input data unit sequences, obtaining N attention distribution information sets and N output data unit sequences, where N is an integer greater than or equal to 1, the large model including multiple processing blocks, the attention distribution information sets including at least one attention distribution information corresponding one-to-one with at least one input data unit in the input data unit sequence, the output data unit sequence including at least one output data unit, the attention distribution information including at least one attention information for the input data unit, the attention information for the input data unit including at least one candidate attention weight, the candidate attention weight being the attention weight between a first data representation of the input data unit in the processing block and a second data representation of the output data unit in the processing block; in response to determining that there is at least one first attention distribution information satisfying a first preset condition among the multiple attention distribution information, determining at least one target invalid data unit based on at least one first input data unit corresponding one-to-one with at least one first attention distribution information, the first preset condition including: the proportion of invalid weights in the attention distribution information being greater than or equal to a preset proportion threshold, the invalid weight being a candidate attention weight less than or equal to a preset weight threshold.

[0005] According to another aspect of this disclosure, a task execution apparatus is provided, comprising: a first execution module, configured to execute N inference tasks of a large model using at least one hardware device based on N input data unit sequences, to obtain N attention distribution information sets and N output data unit sequences, where N is an integer greater than or equal to 1, the large model includes multiple processing blocks, the attention distribution information sets include at least one attention distribution information corresponding one-to-one with at least one input data unit in the input data unit sequence, the output data unit sequence includes at least one output data unit, the attention distribution information includes at least one attention information for the input data unit, the attention information for the input data unit includes at least one candidate attention weight, the candidate attention weight being the attention weight between a first data representation of the input data unit in the processing block and a second data representation of the output data unit in the processing block; and a first determination module, configured to, in response to determining that at least one first attention distribution information satisfying a first preset condition exists among the multiple attention distribution information, determine at least one target invalid data unit based on at least one first input data unit corresponding one-to-one with the at least one first attention distribution information, the first preset condition including: the proportion of invalid weights in the attention distribution information is greater than or equal to a preset proportion threshold, and the invalid weights are candidate attention weights less than or equal to a preset weight threshold.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method provided according to this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the methods provided according to this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to this disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0011] Figure 1 This is a flowchart of a task execution method according to an embodiment of the present disclosure;

[0012] Figure 2 A schematic diagram of attention information according to an embodiment of the present disclosure;

[0013] Figure 3 A schematic flowchart of a task execution method according to an embodiment of the present disclosure;

[0014] Figure 4 A block diagram of a task execution device according to an embodiment of the present disclosure; and

[0015] Figure 5 A block diagram of an electronic device to which the task execution method can be applied according to an embodiment of the present disclosure. Detailed implementation manners

[0016] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0017] The input data received by the large language model may include instructions and queries (also referred to as requests). The instruction can also be called a system prompt or role setting, which can define the global behavior pattern, identity or tone of the model, so that the model will implicitly follow this setting in all subsequent generation processes. For example, if the input text is "You are a smart robot. Please help me solve a math problem", "You are a smart robot" can be an instruction. "Please help me solve a math problem" can be a request.

[0018] However, the instructions input to the model may be invalid. An invalid instruction refers to an instruction that has no effective meaning to the model itself. The model itself does not pay attention to this instruction. This instruction cannot significantly improve the model's ability to follow instructions, and may even damage the model's logical consistency or generalization ability. Invalid instructions may have some characteristics, such as extremely low difficulty and redundancy. After deleting such low-difficulty and redundant data, it has little impact on the model training itself, and may even improve the model's ability. For example, punctuation marks, meaningless auxiliary words "le" and "ne" belong to low-difficulty or redundant data.

[0019] In related technologies, redundant noise with low information content can be removed during invalid data cleaning. However, some instructions with high logical density and high business value are assigned extremely low attention weights during training due to tokenizer bias or model architecture representation limitations. These important instructions are ignored by the model, resulting in substantial training ineffectiveness.

[0020] Furthermore, when a model exhibits extremely low attentional activation towards key logic that should be prioritized for learning (such as complex causal inferences and long-range semantic relationships), it indicates a cognitive bias in the model regarding this type of knowledge. If such high-quality instructions are not recognized, the model will remain in a blind spot, leading to a decline in its capabilities. Even after training the model using one or more different training methods and data, its behavior may still deviate from expectations. This could be because some or all of the key instructions input into the model are not effectively followed or are ignored. The actual reasons for the model's unexpected behavior are difficult to obtain directly through analysis of the model's inference results. Therefore, discovering how the model follows or ignores instructions from an internal perspective may be highly helpful in improving the model's capabilities.

[0021] From an internal model mechanism perspective, the reasons for unexpected model behavior can include: the semantic dilution effect of the tokenizer and the path dependence of the attention mechanism. The semantic dilution effect of the tokenizer refers to the fact that many high-value instructions are fragmented into extremely scattered, low-frequency data units (tokens) during the tokenization stage. This prevents the model from effectively grouping these scattered, low-frequency data units when calculating attention, resulting in semantic degradation. The path dependence of the attention mechanism refers to the fact that models tend to capture simple statistical patterns in the early stages of training. For high-quality instructions that require deep thinking and should have extremely high weights, if the model cannot establish connections in the early stages, it will misclassify them as background noise, forming cognitive inertia and leading to knowledge shielding. It can be understood that when the input data is text, the data units can be called tokens.

[0022] To effectively identify invalid instructions, the underlying logic of how large language models process instruction information is briefly explained below. Large language models consist of multiple cascaded Transformer Blocks. Taking text as an example, at the beginning of the inference phase, the tokenizer parses the original text into a discrete sequence of tokens, which is then mapped to a high-dimensional continuous vector space via an embedding layer. In this process, the vectors combine static semantic representation with positional topological information. Next, processing can be performed based on a self-attention mechanism. As the core of the large language model, the self-attention mechanism's physical essence lies in achieving multi-dimensional dynamic interconnection between vectors. The vectors in the Transformer Block are projected through linear transformations to form a query vector Q, a key vector K, and a value vector V. Global retrieval and dot product operations are used to measure the relevance strength between tokens, yielding attention weights. The multiple weights obtained based on the self-attention mechanism reflect the model's internal resource allocation priority for specific information. The self-attention mechanism can be implemented using the following formula:

[0023] (Formula 1)

[0024] The i-th query vector can and the j-th key vector Attention features between them. V can be a value vector. It can be the dimension of the key vector. It can be the i-th query vector and the j-th key vector Attention weights between them. It is a soft maximum function.

[0025] In theory, attention weights should be highly aligned with the semantic saliency of instructions. However, related techniques still have significant limitations in discovering high-value but ignored invalid instructions. These techniques mainly include hard example mining based on learning dynamics, static screening based on statistical and linguistic features, and screening based on influence functions.

[0026] Hard example mining techniques based on learning dynamics include Forgetting Events and Error L2-Norm (EL2N). These techniques monitor the model's loss convergence trajectory for specific samples during training to evaluate data value. The main logical assumption of this technique is that if an instruction has extremely high logic density, but the model's loss function fails to show a significant decrease in the early stages of training, it is determined that the model has cognitively ignored or is experiencing learning difficulties with that sample.

[0027] The main purpose of the forgetting event technique is to filter out data that is difficult to predict but crucial to the model. The forgetting event technique is defined as follows: if a sample is correctly predicted by the model in one epoch / batch but incorrectly predicted in a later epoch, it is considered forgettable; if a sample is correctly predicted by the model once it has been learned, and is never incorrectly predicted in subsequent training sessions, it is considered unforgettable. Forgettable samples, which cause the model to repeatedly oscillate between correct and incorrect predictions, can be considered difficult and crucial data for training.

[0028] The primary purpose of the L2 norm technique is also to filter out more challenging data for model training, thereby improving model performance. The main definition of the L2 norm technique is: to measure the difficulty of a sample by calculating the L2 norm between the predicted value and the true label. Samples with a higher L2 norm represent data that the model finds confusing or difficult to fit, and can be used for subsequent training to achieve better efficiency than training with the entire dataset.

[0029] Static filtering techniques based on statistical and linguistic features can use indicators such as regular expressions, keyword blacklists, word length distribution, symbol ratio, and perplexity (PPL) to filter invalid data. However, this technique can only identify low-quality data in a literal sense (such as garbled text, advertisements, and duplicate text), and is a coarse-grained filtering method.

[0030] The influence function-based screening technique can determine the change in model parameters after deleting a data point, thereby assessing the importance of the data.

[0031] Techniques such as hard example mining based on learning dynamics, static screening based on statistical and linguistic features, and screening based on influence functions mostly focus on static heuristic filtering, external model scoring, or statistical analysis based on posterior learning dynamics. These techniques suffer from significant timeliness and delays in the decision-making chain. Techniques like forgetting events and second-order error norms heavily rely on real-time feedback after model training begins, essentially constituting post-hoc diagnostics. They cannot provide prior quality insights before deploying computationally intensive training tasks, leading to low iterative efficiency in the data construction phase. Related techniques primarily observe the final results that the model cannot learn through loss fluctuations or error norms, but they cannot extract the reasons for attention mismatch within the instructions, making it difficult to identify genuine low-quality noise or high-value cognitive blind spots caused by word segmentation bias or architectural bottlenecks. Furthermore, static evaluation is often disconnected from model characteristics. Screening mechanisms based on general rules or third-party model scoring ignore the attention allocation characteristics of the model being trained, resulting in many implicit key instructions with deep logical density that should receive high weights being misjudged as invalid and discarded.

[0032] In order to efficiently and accurately identify invalid instructions and reduce the waste of hardware resources by the model based on invalid instructions, this disclosure provides a task execution method, which will be described below.

[0033] Figure 1 This is a flowchart of a task execution method according to an embodiment of the present disclosure.

[0034] like Figure 1 As shown, the method 100 may include operations S110 to S120.

[0035] In operation S110, at least one hardware device is used to perform N inference tasks of a large model based on N input data unit sequences, resulting in N attention distribution information sets and N output data unit sequences.

[0036] In this embodiment of the disclosure, at least one hardware device may be one or more hardware devices in a distributed computing platform. The hardware device may include an artificial intelligence chip. The artificial intelligence chip may be a general-purpose graphics processing unit (GPGPU), a tensor processor (TPU), or a neural network processor (NPU).

[0037] In this embodiment of the disclosure, the input data unit sequence can be obtained based on input data from a large model. The large model can be the large language model described above. The input data can be unimodal or multimodal data including at least one of input text, input image, input audio, and input video. The data unit can be a token. When the input data is input text, the data unit sequence can be a word sequence.

[0038] In this embodiment of the disclosure, N is an integer greater than or equal to 1.

[0039] In this embodiment of the disclosure, the large model includes multiple processing blocks. These processing blocks can be the Transformer blocks described above. The multiple processing blocks can be multiple cascaded processing blocks.

[0040] In this embodiment of the disclosure, the inference task of the large model is used to instruct the large model to generate a response result based on the input data. The response result can also be unimodal or multimodal data. For example, taking input text as an example, the input text can be "Translate: Hello!". The tag of the input text can be "Hello!". It can be understood that the input word sequence of the input text includes more than two words.

[0041] In this embodiment of the disclosure, the input data unit sequence may include at least one input data unit. N attention distribution information sets correspond one-to-one with the N input data unit sequences. Each attention distribution information set includes at least one attention distribution piece of information corresponding one-to-one with at least one input data unit in the input data unit sequence. That is, each piece of attention distribution information corresponds to one input data unit.

[0042] The attention distribution information includes at least one attention information for the input data unit, and the attention information for the input data unit includes at least one candidate attention weight, which is the attention weight between the first data representation of the input data unit in the processing block and the second data representation of the output data unit in the processing block.

[0043] For example, the data representation of an input data unit in a sequence of input data units can differ in different processing blocks. In the first processing block of a large model, the first data representation of the input data unit can be determined based on the embedding vector of that input data unit. In processing blocks following the first processing block, the first data representation of the input data unit can be determined based on an output data representation from a preprocessing block.

[0044] The large model's processing block can process the input sequence based on a multi-head self-attention (MHSA) mechanism. After the large model decodes the l-th output data unit, this l-th output data unit can be concatenated to the current input of the large model to decode the (l+1)-th output data unit, until the end-of-order (EOS) is decoded. l can be an integer greater than or equal to 1. When decoding the end-of-order, the large model's processing block determines the attention weights between the first data representation of the input data unit in this processing block and the second data representation of the already decoded output data unit in this processing block, as candidate attention weights. Candidate attention weights can represent the correlation between the input data unit and the output data unit, or they can represent the contribution of the input data unit to determining the output data unit.

[0045] When a large model comprises multiple processing blocks, the number of attention information entries for the input data unit can be related to the number of processing blocks, or there can be multiple entries. For example, if a large model comprises t processing blocks and each processing block has h attention heads, the number of attention information entries for the input data unit can be t×h. At least one of the multiple attention information entries can be selected to obtain the attention distribution information corresponding to the input data unit.

[0046] In operation S120, in response to determining that at least one first attention distribution information that satisfies a first preset condition exists among a plurality of attention distribution information, at least one target invalid data unit is determined based on at least one first input data unit that corresponds one-to-one with at least one first attention distribution information.

[0047] In this embodiment, the first preset condition includes: the proportion of invalid weights in the attention distribution information is greater than or equal to a preset proportion threshold, where invalid weights are attention weights less than or equal to the preset weight threshold. For example, if an attention weight is less than or equal to the preset weight threshold, that attention weight can be an invalid weight. If an attention weight is greater than the preset weight threshold, that attention weight can be a valid weight. Taking an attention distribution information that includes 100 attention weights and 50 invalid weights among them as an example, the proportion of invalid weights can be 50%. Another example is that the preset weight threshold can be 0.05, and the preset proportion threshold can be 100%. Furthermore, the first input data unit can also be used as the target invalid data unit. It is understood that the preset proportion threshold of 100% is merely an example. For another example, among multiple attention distribution information sets, there may also be a second input attention distribution that does not meet the first preset condition.

[0048] Through the embodiments of this disclosure, one or more attention weights for input data units are obtained during the execution of an inference task, allowing the assessment of the contribution of the input data unit's representation to the output data unit's representation. If a large number of invalid weights exist in an attention distribution, it indicates that the corresponding data unit cannot induce substantial weight activation during inference, thus suggesting that the data unit is marginalized in the semantic representation space of the large model. This data unit is more likely to be an invalid data unit.

[0049] Therefore, invalid data units can be determined based on the actual attention weights generated during the inference process. Based on the model's actual knowledge level, inference ability, and the true inference capability of the model within at least one hardware device, invalid data units relative to the larger model deployed on that hardware device can be objectively and accurately filtered out, thereby filtering out invalid instructions. Furthermore, invalid data units in the input data unit sequence can be removed, saving the computational and storage resources required for attention operations based on these invalid data units, effectively improving hardware resource utilization.

[0050] As can be understood, the method of this disclosure has been described above, and the attention distribution information of this disclosure will be described below.

[0051] In some embodiments, the input data can be input text. The input text can be "Translation: Hello!". This input text is input into the large model to perform an inference task using at least one hardware device on which the large model is deployed, and an output text is obtained. The output text can be, for example, "Hello!". It can be understood that the input text can have tags. The output text is generated by the large model and can be the same as or different from the tag. The process of performing the inference task will be further described below.

[0052] The large model can include a tokenizer. After the input text is input into the large model, the tokenizer can perform token conversion on the input text to obtain multiple tokens. For example, the multiple tokens can include the token of "Translation", the token of ":", the token of "Hello", and the token of "!". It can be understood that different tokenizers have different token conversion methods. Some other tokenizers perform token conversion on this input text and can obtain the tokens of "Fan", "Yi", ":", "Ni", "Hao", and "!". Next, taking the multiple tokens including the token of "Translation", the token of ":", the token of "Hello", and the token of "!" as an example, the description will continue.

[0053] After obtaining multiple tokens, the embedding layer of the large model can map each token to a corresponding vector. Each vector represents the feature representation of a single token in a high-dimensional space. The input text is composed of multiple characters in sequence, so these vectors can also be arranged in the same order to form a vector sequence. This vector sequence can be used as the input to the processing block.

[0054] Inside the processing block, the input is a vector sequence. The self-attention mechanism determines the associations between the vectors in the sequence and generates attention weights. Based on these weights, the model aggregates and transforms the information of the vectors in the sequence and outputs a new vector sequence. The input and output of the processing block are both vector sequences and can have the same length. In addition, each vector in the output sequence of the processing block has already incorporated global context information. After being stacked and processed by multiple processing blocks, the final vector sequence is output by the last processing block.

[0055] The final vector sequence can correspond one-to-one with the input sequence. For example, if the input word sequence contains 4 words, the output can be 4 vectors. At this point, the first decoded word has not yet been generated; it will be predicted from the last vector in the vector sequence. For example, the model can extract the last vector in the vector sequence (corresponding to the position of the "!" at the end of the input sequence). This vector aggregates the semantic information of all preceding words and can be used as the basis for predicting the "next word". This vector can be mapped to a logits vector through a linear transformation and then converted into a probability distribution through a soft maximum function. Based on this probability distribution, the model can select the word with the highest probability as the prediction result; for example, the word "Hello" can be the first decoded word.

[0056] During the autoregressive generation process, the generated tokens (the tokens for "Hello") are converted into new vectors and appended to the end of the original input vector sequence to form a new input sequence. This expanded sequence is then processed by the model again to predict the next token until an end marker is generated or a preset length is reached, ultimately combining to form the complete output text (e.g., "Hello!").

[0057] The following will combine Figure 2 The attention weight matrix of this disclosure will be further explained.

[0058] Figure 2 This is a schematic diagram of attention information according to an embodiment of the present disclosure.

[0059] like Figure 2 As shown, after processing by the word segmenter, multiple input word units can be obtained from the input text "Translate: Hello!". These multiple input word units include the word unit for "translate", the word unit for ":", the word unit for "hello", and the word unit for "!". After the large model decodes the end-of-line character, multiple output word units can be obtained. These multiple output word units include the word unit for "Hello" and the word unit for "!". It can be understood that the output text for this input text can be "Hello!".

[0060] Large model inference can include multiple processing blocks. After the inference task is completed, the attention weight matrix of a processing block can be obtained. During the decoding process of decoding the end symbol, the input vector sequence received by this processing block can include the data representation of "translation", the data representation of ":", the data representation of "hello", the data representation of the input "!", the data representation of "Hello", and the data representation of the decoded "!". It can be understood that the data representation of "translation", the data representation of ":", the data representation of "hello", and the data representation of the input "!" are also the data representations of the input tokens in this processing block and can be used as the first data representations respectively. The data representation of "Hello" and the data representation of "!" are also the data representations of the output tokens in this processing block and can be used as the second data representations respectively.

[0061] The attention weight matrix am20 includes multiple attention information. Among the multiple attention information, the attention information for "translation" includes the attention weight "0.2" between the data representation of "translation" and the data representation of "Hello", and the attention weight "0.1" between the data representation of "translation" and the data representation of the decoded "!". Among the multiple attention weights, the attention information for ":" includes the attention weight "0.01" between the data representation of ":" and the data representation of "Hello", and the attention weight "0.00" between the data representation of ":" and the data representation of "!". The attention information for "hello" includes the attention weight "0.7" between the data representation of "hello" and the data representation of "Hello", and the attention weight "0.1" between the data representation of "hello" and the data representation of "!". The attention information for the input "!" includes the attention weight "0.04" between the data representation of "!" and the data representation of "Hello", and the attention weight "0.6" between the data representation of the input "!" and the data representation of the output "!".

[0062] In some embodiments, the number of attention information for input data units can be set.

[0063] For example, if the number of attention information for input data units is set to 1, a processing block can be randomly selected, and the corresponding attention information can be obtained as the attention distribution information. Taking Figure 2 the attention information for ":" shown as an example, this attention information can be used as the attention distribution information. If the preset weight threshold is 0.05 and the preset proportion threshold is 1, both of the 2 attention weights in this attention distribution information are less than the preset weight threshold, and the invalid weight proportion (1) is equal to the preset proportion threshold. This attention distribution information meets the first preset condition and can be used as the first attention distribution information. The token of ":" can be used as the first input data unit.

[0064] For example, if the number of attention information points for an input data unit is set to three, three processing blocks can be selected to obtain the corresponding attention information as attention distribution information. Taking the attention distribution information for ":" as an example, this attention distribution information can include three attention information points. One of these three attention information points can be... Figure 2 The attention information shown is for the colon ":". If the preset weight threshold is 0.05, the preset proportion threshold is 1, and there are 4 attention weights in the attention distribution information that are greater than the preset weight threshold, and the proportion of invalid weights (2 / 3) is less than the preset proportion threshold, then the attention distribution information does not meet the first preset condition. The colon ":" can be used as the second input data unit or as a valid data unit. It can be understood that the attention distribution information set corresponding to the input colon sequence can include the attention distribution information for the colon "translate", the attention distribution information for the colon ":", the attention distribution information for the colon "hello", and the attention distribution information for the colon "!".

[0065] It is understandable that the first input data unit can be used as the target invalid data unit, or the target invalid data unit can be selected from one or more first input data units, which will be explained below.

[0066] It's understandable, based on the above. Figure 2 The attention information of this disclosure has been explained, and the method of this disclosure will be further explained below.

[0067] Figure 3 This is a schematic flowchart of a task execution method according to an embodiment of the present disclosure.

[0068] like Figure 3 As shown, method 300 may include operations S311, S312, and operations S321 to S327.

[0069] In operation S311, at least one hardware device is used to perform an inference task of a large model based on the input data unit sequence, resulting in an attention distribution information set and an output data unit sequence. The attention distribution information set includes at least one attention distribution information corresponding one-to-one with at least one input data unit in the input data unit sequence.

[0070] In operation S312, it is determined whether the attention distribution information meets the first preset condition. As mentioned above, the first preset condition includes that the proportion of invalid weights in the attention distribution information is greater than or equal to a preset proportion threshold. Taking a preset proportion threshold of 1 and a preset weight threshold of 0.05 as an example, in response to multiple candidate attention weights in the attention distribution information being less than or equal to 0.05, operation S321 can be executed. In response to one of the multiple candidate attention weights in the attention distribution information being greater than 0.05, operation S322 can be executed.

[0071] In operation S321, a first input data unit corresponding to the first attention distribution information that satisfies the first preset condition is determined. The first input data unit may be a factually invalid data unit, an irrationally invalid data unit, or a low-frequency redundant and negligible data unit.

[0072] In operation S322, the second input data unit corresponding to the second attention distribution information that does not meet the first preset condition is determined as a valid data unit.

[0073] It is understandable that operations S311 to S312 can be executed N times to obtain N inference tasks executed in parallel or serially. The sequence of N input data units used when executing N inference tasks includes M input data units. Taking N greater than 1 and multiple first input data units as an example, there may be identical data units among the multiple first input data units, that is, at least one of the multiple first input data units appears multiple times in the N inference processes, and the proportion of invalid weights in the corresponding attention distribution information is relatively large.

[0074] Next, for a plurality of first input data units, operations S323 to S327 can be performed to determine at least one target invalid data unit based on at least one first input data unit.

[0075] In operation S323, at least one candidate invalid data unit and invalid frequency information of the candidate invalid data unit are determined based on a plurality of first input data units.

[0076] Multiple first input data units can be deduplicated to obtain at least one candidate invalid data unit.

[0077] Invalid frequency information can be the High-frequency Ineffectiveness Ratio (HIR). The invalid frequency information can be determined based on a first number and a second number of candidate invalid data units. The first number is the number of candidate invalid data units among multiple first input data units, and the second number is the number of candidate invalid data units among M input data units. For example, if a data unit appears 10 times in M ​​input data units, the first number can be 8. This data unit can be considered a candidate invalid data unit, and the corresponding invalid frequency information can be determined by dividing the first number by the second number, for example, 0.8. As another example, if another data unit appears 20 times in M ​​input data units, the second number can be 20. If this data unit appears only once among multiple first input data units, the first number can be 1. This data unit can be considered a candidate invalid data unit, and the corresponding invalid frequency information can be determined by dividing the first number by the second number, for example, 0.05.

[0078] Next, based on at least one of type information and at least one invalid frequency information, a target invalid data unit is determined from at least one candidate invalid data unit. The type information indicates the type of the candidate invalid data unit. Taking an invalid lexical as an example, the type information of the lexical can indicate that the lexical is a punctuation mark, particle, verb, noun, or logical connector. Logical connectors can be "and," "or," etc. This will be explained below in conjunction with operations S324 to S327.

[0079] In operation S324, determine whether the invalid frequency information is greater than or equal to the preset invalid frequency threshold.

[0080] In response to determining that the invalid frequency information of the candidate invalid data unit is greater than or equal to a preset invalid frequency threshold, operation S325 is executed. In operation S325, it is determined whether the candidate invalid data unit meets a second preset condition.

[0081] The second presupposition condition includes at least one of the following: the input validity information of the candidate invalid data unit indicates that the candidate invalid data unit is valid; the type information of the candidate invalid data unit indicates that the candidate invalid data unit is a verb, logical connector, or noun. The input validity information can be input by the user or an expert and can indicate whether the candidate invalid data unit is valid or invalid. Furthermore, if the type of the candidate invalid data unit is a verb, logical connector, or noun, it indicates that the data unit may have actual semantic meaning, but it has not been effectively captured by the model and is therefore unreasonably invalid.

[0082] In response to the candidate invalid data unit not meeting the second preset condition, operation S326 is executed. In operation S326, the candidate invalid data unit is determined as the target invalid data unit. If the type information of the candidate invalid data unit indicates that the candidate invalid data unit is a punctuation mark or particle, the candidate invalid data unit is a low information entropy data unit, which can be determined as natural and negligible natural noise with natural character semantics, belonging to factual invalid data units. Furthermore, if the input validity information indicates that the data unit is invalid, the candidate invalid data unit can be directly determined as the target invalid data unit.

[0083] In response to a candidate invalid data unit meeting the second preset condition, operation S327 is executed. In operation S327, the candidate invalid data unit is determined as a valid data unit. If the type information of the candidate invalid data unit indicates that it is a verb, logical connector, or noun, this candidate invalid data unit is an unreasonable invalid data unit, resulting from insufficient model capability or the hardware resources of the hardware device being unable to fully utilize the model's performance. Furthermore, if the input validity information indicates that the data unit is valid, the candidate invalid data unit can be directly determined as a valid data unit. Additionally, some core verbs and key entity nouns can be preset to treat some verbs or unimportant nouns as invalid data units, further saving hardware resource overhead.

[0084] It is understood that the above description uses the example of invalid frequency information being greater than a preset invalid frequency threshold to illustrate this disclosure. However, this disclosure is not limited to this; invalid frequency information can also be less than the preset invalid frequency threshold, as will be explained below.

[0085] In response to the determination that the invalid frequency information of the candidate invalid data unit is less than a preset invalid frequency threshold, the above operation S326 is performed. For example, when the invalid frequency information is less than the preset invalid frequency threshold, the candidate invalid data unit can be regarded as a low-frequency, redundant, and negligible data unit, and can be used as the target invalid data unit.

[0086] This disclosure overcomes the lag inherent in posterior statistics, enabling a priori determination of instruction validity. Related techniques, such as forgetting event techniques and second-order error norms, belong to the posterior evaluation paradigm and rely on statistical analysis of the loss function trajectory after model training begins. However, training requires significant hardware resources. Before investing substantial computing power, developers cannot predict the validity of constructed data based on posterior statistical techniques, resulting in a significant time lag. This disclosure achieves a priori instruction validity evaluation by extracting the micro-attention distribution during the inference phase. Invalid instructions can be accurately identified during data preprocessing or inference without undergoing a complete training cycle, thus avoiding wasted computing resources due to cognitive idling and significantly improving the iterative efficiency of data engineering.

[0087] As we have explained some methods for identifying target invalid data units above, we will now explain methods for improving hardware resource utilization based on target invalid data units.

[0088] In some embodiments, the method may further include: deleting target invalid data units from the input data unit sequence to obtain a first updated data unit sequence. After deleting invalid data units, each processing block of the large model can reduce a significant amount of attention operations during inference or training, effectively reducing hardware resource overhead such as computation and storage. This results in reduced hardware resource overhead when performing inference or training tasks of the large model based on the first updated data unit sequence, while maintaining or even improving the accuracy of inference or training.

[0089] It is understood that the above description has explained the method for deleting invalid data units. However, this disclosure is not limited to this; the input data may include instructions and requests. Accordingly, the sequence of input data units may include a subsequence of input instruction data units and a subsequence of input query data units. If the number of invalid data units in the subsequence of input instruction data units is large, the subsequence of input instruction data units may be deleted, as will be explained below.

[0090] In some embodiments, the method may further include: determining the input instruction data unit subsequence as an invalid instruction data unit subsequence in response to determining that the number of target invalid data units in the input instruction data unit subsequence is greater than or equal to a preset number threshold; deleting the invalid instruction data unit subsequence from the input data unit sequence to obtain a second updated data unit sequence; and performing inference or training tasks based on the second updated data unit sequence. During the execution of inference or training tasks, with the input instruction data unit subsequence deleted, the number of data units that the model needs to process is significantly reduced, each processing block can reduce a large amount of attention computation, effectively reducing hardware resource overhead such as computing and storage, thus reducing hardware resource overhead when performing inference or training tasks of large models based on the second updated data unit sequence.

[0091] It is understandable that if the number of invalid weights in an instruction is large, it is difficult to induce substantial weight activation during inference. This instruction is marginalized in the model's semantic representation space. When the model can maintain consistency with the output target even in the absence of such lexical features, this instruction sequence can be considered invalid relative to the model's current cognitive level.

[0092] It is understandable that deleting one or more data units can be achieved by deleting characters from input text, image blocks from input images, or audio segments from input audio.

[0093] In some embodiments, the input instruction data unit subsequence is an input instruction word subsequence, which is determined based on the input instruction subtext in the input data. The instruction can be in text form, and the corresponding input instruction data unit subsequence can be an input instruction word subsequence.

[0094] As can be understood, the above text has explained the processing method for input sequences containing target invalid data units. The following text will explain the candidate invalid data units that meet the second preset condition.

[0095] In some embodiments, the above method may further include:

[0096] In response to the candidate invalid data unit satisfying the second preset condition, at least one training data unit sequence is obtained based on the candidate invalid data unit.

[0097] At least one training data unit sequence includes at least one of at least a first training data unit sequence and at least one second training data unit sequence, wherein the first training data unit sequence is an input data unit sequence including candidate invalid data units, and the second training data unit sequence is generated based on the candidate invalid data units. A sequence including candidate invalid data units that satisfy a second preset condition can be used for training. Alternatively, a new sequence can be generated based on the candidate invalid data units that satisfy the second preset condition for training.

[0098] By utilizing at least one hardware device to perform a large model training task based on an input data unit sequence including candidate invalid data units, at least one attention weight in the attention distribution information corresponding to the candidate invalid data units is increased. Thus, the training objective includes increasing the attention weights of the candidate invalid data units, thereby effectively improving the model's cognitive and reasoning abilities.

[0099] Through the embodiments of this disclosure, data synthesis can be guided based on candidate invalid data units that meet a second preset condition. By introducing a high-frequency invalid offset rate (HIR), an evaluation mechanism is established between objective semantic value and the model's subjective perception. This mechanism can identify high-quality instructions that humans consider important but are ignored by the model, and then perform targeted synthesis based on these ignored high-quality instructions to train the model and improve its capabilities.

[0100] As can be understood, the method of this disclosure has been described above, and the input data of this disclosure will be described below.

[0101] In some embodiments, the input data unit sequence is determined based on at least one of input text, input image, input audio, and input video, and the output data unit sequence is used to convert to at least one of output text, output image, output audio, and output video.

[0102] For example, the input data unit sequence includes an input word sequence determined based on the input text, and the output data unit sequence is an output word sequence. Performing a large-scale model inference task using at least one hardware device based on the input data unit sequence includes: performing word transformation on the input text to obtain an input word sequence; performing a large-scale model inference task using at least one hardware device based on the input word sequence to obtain an output word sequence for the input word sequence and at least one attention distribution information corresponding one-to-one with at least one input word in the input word sequence; and obtaining the output text based on the output word sequence.

[0103] For example, the input data unit sequence includes an input image data unit sequence determined based on the input image, and the output data unit sequence is an output image data unit sequence. Performing a large model inference task using at least one hardware device based on the input data unit sequence includes: performing patch embedding on the input image to obtain the input image data unit sequence; performing a large model inference task using at least one hardware device based on the input image data unit sequence to obtain an output image data unit sequence for the input image data unit sequence and at least one attention distribution information corresponding one-to-one with at least one input image data unit in the input image data unit sequence; and obtaining the output image based on the output image data unit sequence.

[0104] Through the embodiments disclosed herein, implicit high-value data can be mined to improve model capabilities. For example, invalid instructions can be extracted from massive amounts of data to identify model weaknesses, especially those of models deployed on current hardware devices. Model capabilities can be improved by specifically targeting these ignored instructions with semantic reconstruction, CoT (Coding of Thought) guidance, or sampling weight compensation. For instance, instructions with extremely low attention weights but complex semantics can be defined as cognitive difficulties, and strategies such as learning can be used to force the model to increase its attention to them during subsequent training, thereby specifically improving model capabilities.

[0105] This disclosure establishes a complete and objective data production process feedback loop. By analyzing the model's attention distribution on specific instructions, model weaknesses are identified, and targeted training data is synthesized to guide data production. A feedback-based data evolution pipeline can be constructed, transforming identified invalid instructions (i.e., high-value but overlooked semantic regions) into sampling guidelines for synthetic data. For cognitive weaknesses exhibited by the model, this type of data is synthesized more frequently. Therefore, it ensures that the synthesized data is not merely low-quality repetition, but rather an effective expansion of the model's cognitive boundaries.

[0106] It is understood that the method of this disclosure has been described above, and the apparatus of this disclosure will be described below.

[0107] Figure 4 This is a schematic diagram of a task execution apparatus according to an embodiment of the present disclosure.

[0108] like Figure 4 As shown, the device 400 includes a first execution module 410 and a first determination module 420.

[0109] The first execution module 410 is used to execute N inference tasks of a large model based on N input data unit sequences using at least one hardware device, to obtain N attention distribution information sets and N output data unit sequences. N is an integer greater than or equal to 1, and the large model includes multiple processing blocks. The attention distribution information sets include at least one attention distribution information corresponding one-to-one with at least one input data unit in the input data unit sequence. The output data unit sequence includes at least one output data unit. The attention distribution information includes at least one attention information for the input data unit. The attention information for the input data unit includes at least one candidate attention weight, which is the attention weight between the first data representation of the input data unit in the processing block and the second data representation of the output data unit in the processing block.

[0110] The first determining module 420 is configured to, in response to determining that at least one first attention distribution information satisfying a first preset condition exists among a plurality of attention distribution information, determine at least one target invalid data unit based on at least one first input data unit corresponding one-to-one with at least one first attention distribution information. The first preset condition includes: the proportion of invalid weights in the attention distribution information is greater than or equal to a preset proportion threshold, and the invalid weights are candidate attention weights that are less than or equal to a preset weight threshold.

[0111] In some embodiments, there are at least one first input data unit, and the multiple input data units contain the same data unit. The sequence of N input data units includes M input data units. The first determining module 420 includes: an obtaining submodule, configured to obtain at least one candidate invalid data unit and invalidity frequency information of the candidate invalid data unit based on the multiple first input data units. The invalidity frequency information is determined based on a first quantity and a second quantity of candidate invalid data units, where the first quantity is the number of candidate invalid data units in the multiple first input data units, and the second quantity is the number of candidate invalid data units in the M input data units. The first determining submodule is configured to determine a target invalid data unit from the at least one candidate invalid data unit based on at least one of at least one type information and at least one invalidity frequency information. The type information is used to indicate the type of the candidate invalid data unit.

[0112] In some embodiments, the first determining submodule includes at least one of the following: a first determining unit, configured to determine a candidate invalid data unit as a target invalid data unit in response to determining that the invalid frequency information of the candidate invalid data unit is greater than or equal to a preset invalid frequency threshold and that the candidate invalid data unit does not meet a second preset condition; and a second determining unit, configured to determine a candidate invalid data unit as a target invalid data unit in response to determining that the invalid frequency information of the candidate invalid data unit is less than the preset invalid frequency threshold. The second preset condition includes at least one of the following: the input validity information of the candidate invalid data unit indicates that the candidate invalid data unit is valid; the candidate invalid frequency information is greater than or equal to the first preset invalid frequency threshold and the type information of the candidate invalid data unit indicates that the candidate invalid data unit is a verb, a logical link word, or a noun.

[0113] In some embodiments, the apparatus 400 may further include: a first deletion module, configured to delete target invalid data units in the input data unit sequence to obtain a first updated data unit sequence, thereby reducing the hardware resource overhead when performing inference or training tasks of a large model based on the first updated data unit sequence.

[0114] In some embodiments, the input data unit sequence includes an input instruction data unit subsequence and an input query data unit subsequence. The apparatus 400 may further include: a second determining module, configured to determine the input instruction data unit subsequence as an invalid instruction data unit subsequence in response to determining that the number of target invalid data units in the input instruction data unit subsequence is greater than or equal to a preset number threshold; and a second deleting module, configured to delete the invalid instruction data unit subsequence from the input data unit sequence to obtain a second updated data unit sequence, thereby reducing the hardware resource overhead when performing inference or training tasks on a large model based on the second updated data unit sequence.

[0115] In some embodiments, the input instruction data unit subsequence is an input instruction word subsequence, which is determined based on the input instruction subtext in the input data.

[0116] In some embodiments, the apparatus 400 may further include: an obtaining module, configured to, in response to a candidate invalid data unit satisfying a second preset condition, obtain at least one training data unit sequence based on the candidate invalid data unit, the at least one training data unit sequence including at least one first training data unit sequence and at least one second training data unit sequence, the first training data unit sequence being an input data unit sequence including the candidate invalid data unit, and the second training data unit sequence being generated based on the candidate invalid data unit; and a second execution module, configured to, using at least one hardware device, perform a training task of a large model based on the input data unit sequence including the candidate invalid data unit, to increase at least one attention weight in the attention distribution information corresponding to the candidate invalid data unit.

[0117] In some embodiments, the input data unit sequence is determined based on at least one of input text, input image, input audio, and input video, and the output data unit sequence is used to convert to at least one of output text, output image, output audio, and output video.

[0118] In some embodiments, the input data unit sequence includes an input word sequence determined based on the input text, and the output data unit sequence is an output word sequence. The first execution module 410 includes: a word conversion submodule for performing word conversion on the input text to obtain an input word sequence; an execution submodule for using at least one hardware device to perform an inference task of a large model based on the input word sequence, obtaining an output word sequence for the input word sequence and at least one attention distribution information corresponding one-to-one with at least one input word in the input word sequence; and an obtaining submodule for obtaining the output text based on the output word sequence.

[0119] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0120] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0121] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0122] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0123] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0124] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as task execution methods. For example, in some embodiments, the task execution method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the task execution method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform a task execution method by any other suitable means (e.g., by means of firmware).

[0125] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0126] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0127] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM) or flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0128] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) monitor or a liquid crystal display (LCD)); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0129] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0130] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0131] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0132] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A task execution method, comprising: Using at least one hardware device, N inference tasks of a large model are executed based on N input data unit sequences to obtain N attention distribution information sets and N output data unit sequences, where N is an integer greater than or equal to 1. The large model includes multiple processing blocks. The attention distribution information sets include at least one attention distribution information corresponding one-to-one with at least one input data unit in the input data unit sequence. The output data unit sequence includes at least one output data unit. The attention distribution information includes at least one attention information for the input data unit. The attention information for the input data unit includes at least one candidate attention weight. The candidate attention weight is the attention weight between the first data representation of the input data unit in the processing block and the second data representation of the output data unit in the processing block. In response to determining that at least one first attention distribution information among the plurality of attention distribution information satisfies a first preset condition, at least one target invalid data unit is determined based on at least one first input data unit corresponding one-to-one with at least one first attention distribution information. The first preset condition includes: the proportion of invalid weights in the attention distribution information is greater than or equal to a preset proportion threshold, and the invalid weights are candidate attention weights that are less than or equal to a preset weight threshold.

2. The method according to claim 1, wherein at least one of the first input data units is multiple, and the multiple input data units contain the same data unit, and the sequence of N input data units includes M input data units. in, The step of determining at least one target invalid data unit based on at least one first input data unit corresponding one-to-one with at least one first attention distribution information includes: Based on a plurality of first input data units, at least one candidate invalid data unit and invalidity frequency information of the candidate invalid data unit are obtained. The invalidity frequency information is determined based on a first quantity and a second quantity of the candidate invalid data units, wherein the first quantity is the number of candidate invalid data units in the plurality of first input data units, and the second quantity is the number of candidate invalid data units in M ​​input data units. A target invalid data unit is determined from at least one of the candidate invalid data units based on at least one of type information and at least one invalid frequency information, wherein the type information is used to indicate the type of the candidate invalid data unit.

3. The method according to claim 2, wherein, Determining the target invalid data unit from at least one of the candidate invalid data units includes at least one of the following: In response to determining that the invalid frequency information of the candidate invalid data unit is greater than or equal to a preset invalid frequency threshold, and determining that the candidate invalid data unit does not meet the second preset condition, the candidate invalid data unit is determined as the target invalid data unit; In response to determining that the invalidity frequency information of the candidate invalid data unit is less than a preset invalidity frequency threshold, the candidate invalid data unit is determined as the target invalid data unit. The second preset condition includes at least one of the following: The input validity information of the candidate invalid data unit indicates that the candidate invalid data unit is valid; The candidate invalid frequency information is greater than or equal to the first preset invalid frequency threshold, and the type information of the candidate invalid data unit indicates that the candidate invalid data unit is a verb, logical link word, or noun.

4. The method according to claim 3, further comprising: Deleting the target invalid data unit from the input data unit sequence yields a first updated data unit sequence, thereby reducing the hardware resource overhead when performing the inference or training task of the large model based on the first updated data unit sequence.

5. The method according to claim 3, wherein, The input data unit sequence includes an input instruction data unit subsequence and an input query data unit subsequence. Also includes: In response to determining that the number of target invalid data units in the input instruction data unit subsequence is greater than or equal to a preset number threshold, the input instruction data unit subsequence is determined to be an invalid instruction data unit subsequence. The invalid instruction data unit subsequence in the input data unit sequence is deleted to obtain the second updated data unit sequence, thereby reducing the hardware resource overhead when performing the inference or training task of the large model according to the second updated data unit sequence.

6. The method according to claim 5, wherein, The input instruction data unit subsequence is an input instruction word subsequence, which is determined based on the input instruction subtext in the input data.

7. The method according to claim 2, wherein, Also includes: In response to the candidate invalid data unit satisfying a second preset condition, at least one training data unit sequence is obtained based on the candidate invalid data unit. The at least one training data unit sequence includes at least one of a first training data unit sequence and at least one second training data unit sequence. The first training data unit sequence is an input data unit sequence including the candidate invalid data unit, and the second training data unit sequence is generated based on the candidate invalid data unit. The large model is trained using at least one hardware device based on a sequence of input data units including the candidate invalid data units, in order to increase at least one attention weight in the attention distribution information corresponding to the candidate invalid data units.

8. The method according to claim 1, wherein, The input data unit sequence is determined based on at least one of input text, input image, input audio, and input video, and the output data unit sequence is used to convert to at least one of output text, output image, output audio, and output video.

9. The method according to claim 8, wherein, The input data unit sequence includes an input word sequence determined based on the input text, and the output data unit sequence is an output word sequence. The step of performing the inference task of the large model using at least one of the hardware devices based on the input data unit sequence includes: The input text is subjected to lexical transformation to obtain an input lexical sequence; The large model's inference task is performed using at least one of the hardware devices based on the input word sequence, to obtain the output word sequence for the input word sequence and at least one attention distribution information corresponding one-to-one with at least one input word in the input word sequence; The output text is obtained based on the output word sequence.

10. A task execution device, comprising: The first execution module is used to execute N inference tasks of a large model based on N input data unit sequences using at least one hardware device, to obtain N attention distribution information sets and N output data unit sequences, where N is an integer greater than or equal to 1. The large model includes multiple processing blocks. The attention distribution information sets include at least one attention distribution information corresponding one-to-one with at least one input data unit in the input data unit sequence. The output data unit sequence includes at least one output data unit. The attention distribution information includes at least one attention information for the input data unit. The attention information for the input data unit includes at least one candidate attention weight. The candidate attention weight is the attention weight between the first data representation of the input data unit in the processing block and the second data representation of the output data unit in the processing block. The first determining module is configured to, in response to determining that at least one first attention distribution information satisfying a first preset condition exists among a plurality of attention distribution information, determine at least one target invalid data unit based on at least one first input data unit corresponding one-to-one with at least one first attention distribution information. The first preset condition includes: the proportion of invalid weights in the attention distribution information is greater than or equal to a preset proportion threshold, and the invalid weights are candidate attention weights that are less than or equal to a preset weight threshold.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images