451 results about "Mask" patented technology

The invention relates to the technical field of artificial intelligence chips, and provides an operator optimization method, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining the block size of operator data in each dimension based on the batch size and mask mode of the operator data and the hardware parameters of computing equipment, each dimension comprises a batch dimension and a sequence length dimension; segmenting the operator data based on the block size of the operator data in each dimension to obtain a plurality of data blocks; and distributing the calculation tasks corresponding to the plurality of data blocks to a plurality of processing units on the calculation equipment, and performing parallel execution on the calculation tasks corresponding to the plurality of data blocks based on the plurality of processing units. According to the method and the device, the data are segmented in the batch dimension and the sequence length dimension at the same time, so that when the data batch is small, a plurality of processing units on the computing equipment can also participate in computing at the same time, hardware resources are prevented from being idle, the utilization rate of the hardware resources is improved, and the overall computing efficiency is improved.

The invention provides an optimization method for improving parallelism and performance of hardware executing RISC-V vector instructions, which comprises the following steps: S1, an instruction fetching and decoding stage step: the step comprises an instruction fetching link and a decoding link, and the instruction fetching link reads instructions from a memory according to a program sequence and stores the instructions into an instruction queue; the decoding link comprises the steps of analyzing an instruction, identifying whether the instruction is a vector mask instruction or a vector length control instruction, and if the instruction needs an old value of a destination register, marking that the instruction needs to carry an old value dependency identifier, belonging to the technical field of optimization methods. According to the method, the instruction is transmitted without waiting for the old value of the target vector register to be ready, the instruction can be transmitted in advance after a certain condition is met, and the correct old value is obtained, so that the aim of improving the parallelism and the performance of hardware for executing the RISC-V vector instruction is fulfilled.

The invention discloses an accelerator for accelerating the reasoning process of an unstructured sparse large language model and an operation method of the accelerator. The accelerator comprises a processor module, a direct memory access module, a memory controller module, a DDR memory module, an interconnection bus module, a calculation core module and a router module. The interconnection bus module is connected with the direct memory access module, the memory controller module and the computing core module; the processor module is connected with the direct memory access module; the DDR memory module is connected with the memory controller module; the router module is connected with the computing core module and the adjacent router module; and the calculation core module obtains to-be-calculated data from the off-chip DDR memory module through the interconnection bus module. According to the method, the limit that data reuse is carried out only along a single row or a single column on a traditional systolic array is broken through, cross-row and cross-column data reuse is achieved, the pressure of hardware equipment is relieved, the positions of non-zero elements are coded through bit masks, and the calculation efficiency is improved.

Techniques are disclosed that relate to executing pairs of instructions. A processor may include fusion detector circuitry configured to detect a pair of fetched instructions and fuse the pair of fetched instructions into a fused instruction operation, and execution circuitry coupled to the fusion detector circuitry and configured to execute the fused instruction operation. In some embodiments the pair of instructions is executable to generate a remainder of a division operation. In some embodiments the pair of instructions is executable to compare two operands and perform a write operation based on the comparison. In some embodiments the pair of instructions is executable to perform an operation and apply a mask bit sequence to the result. The fusion detector circuitry may also be configured to obtain first and second portions of a constant value from first and second instructions and store the first and second portions in a destination register.

Some embodiments perform fully unsupervised string error detection and repair, for strings containing both syntactic errors and semantic errors. In some embodiments, string error mitigation includes learning a regular-expression-based pattern that characterizes at least a predefined fraction of a set of string data values, identifying and masking a semantic substring, deriving edits to repair a string which contains the semantic substring, and replacing a mask value with a replacement substring that conforms to the pattern. Some embodiments utilize a language model. Some embodiments include or utilize a pattern learner, one or more learned significant patterns, and a repair engine. Some embodiments exploit program execution information to generate patterns when significant patterns are not initially found.

A generative language model (e.g. large language model) may be used to generate programming language code. However, the generative language model may sometimes generate an output that is not compliant with the grammar of the programming language. In some embodiments herein, a generative language model may be modified to only generate an output that is grammar-compliant. A method may include generating a plurality of values using the generative language model, where each of the values is indicative of a probability of a respective token being a next token of a token sequence generated by the generative language model. A mask may be applied to the plurality of values. The mask may operate on each value that corresponds to a token not compliant with the grammar of the programming language to reduce or zero the probability of that token being the next token.

One problem of a generative language model (e.g. a large language model) is the generation of syntactically-invalid or misinformed output. This may be mitigated by utilizing a grammar defining valid sequences of output. The grammar may constrain the token generation. A method may include obtaining values generated using the generative language model, where each value is indicative of a probability of a respective token being a next token in the token sequence. The method may further include obtaining a mask based on the token sequence already generated and the grammar. The method may further include applying the mask to the values. The mask may operate on each value that corresponds to a token not compliant with the grammar to reduce or zero the probability of the corresponding token being the next token. The next token is then determined based on the values after the mask is applied.

Systems and methods are provided that train a machine-learned language encoding model through the use of a contrastive learning task. In particular, the present disclosure describes a contrastive learning task where the encoder learns to distinguish input tokens from plausible alternatives. In some implementations, on each training example the proposed method masks out some subset (e.g., 15%) of the original input tokens, replaces the masked tokens with samples from a “generator” (e.g., which may be a small masked language model), and then trains the encoder to predict whether each token comes from the original data or is a replacement produced by the generator.

The invention relates to the technical field of electric digital data processing, and discloses a multi-dimensional data logic processing method based on an artificial intelligence algorithm, which comprises the following steps: receiving a target discrete data packet, extracting metadata, and calculating to generate a dimension entropy feature vector representing data logic complexity; inputting the vector into a preset topological mapping model, and outputting an initial adjacent matrix; calling a feedback suppression mask matrix generated based on a historical operator utility state, and executing bitwise logic AND operation with the initial adjacent matrix to generate a corrected effective topological matrix; the matrix is analyzed, a logic operator function pointer is dynamically indexed in an instruction cache, and a directed acyclic execution linked list is constructed; according to the method, redundant logic nodes in AI prediction are definitely eliminated through a bit operation mask mechanism based on historical feedback, and deterministic convergence of processing delay and optimal matching of computing power resources are achieved.

The invention relates to the technical field of security assessment of cryptographic algorithms, and discloses a security assessment method for parallel implementation of mask circuits, computer equipment, a computer readable storage medium and a computer program product in order to solve the problems of how to assess the security of parallel implementation of the mask circuits and how to reduce the security assessment measurement times. The method comprises the following steps: acquiring electromagnetic leakage information and differential energy attack parameters of each share in a mask circuit realized by adopting parallelization; according to the electromagnetic leakage information of each share, calculating the leakage signal-to-noise ratio of each share; under the condition that the signal-to-noise ratio meets a stable condition, calculating a success rate of recovering a correct secret key under the single-variable high-order differential energy attack according to a cryptographic algorithm related parameter, the signal-to-noise ratio of each share leakage and a differential energy attack parameter; and according to the success rate, evaluating the safety of the mask circuit realized by adopting parallelization. By adopting the method, the success rate can be quickly and accurately calculated, and the number of times of measurement required by safety assessment is reduced.

The invention particularly discloses a model compression method based on a quantized large language model, and relates to the technical field of large language model compression. The method comprises the following steps: S1, performing quantization processing on a pre-trained large language model to obtain a quantized weight matrix, and calculating a pruning optimization objective function after quantization; s2, designing a post-quantization pruning measurement index based on the quantized weight matrix, and combining the pruning measurement index with an absolute value of the quantized weight and a pruning measurement regularization result of the original weight; and S3, based on a pruning measurement index, carrying out importance sorting on the quantized weight, generating a dynamic binary pruning mask, and minimizing quantization and pruning errors through a block optimization algorithm. The method adopts a block quantization post-pruning algorithm, minimizes quantization and pruning errors, and maximizes the model compression ratio on the premise of ensuring the model performance so as to meet the requirements of actual application on the model.

The invention relates to a large model knowledge distillation method and device for a Text2SQL task, and the method comprises the steps: carrying out the supervision training of a target large language model based on an original Text2SQL training sample, so as to generate a teacher model corresponding to the Text2SQL task; random mask operation is carried out on SQL statements in the original Text2SQL training sample to obtain a mask sequence, the mask sequence is input into the initial student model to predict mask lexical elements at mask positions in the mask sequence to obtain mask prediction lexical elements, the mask lexical elements in the mask sequence are replaced with the corresponding mask prediction lexical elements, and the mask prediction lexical elements in the original Text2SQL training sample are obtained. A noise adding sample is constructed; and determining a target training sample according to the noise adding sample, and on the basis of the target training sample, combining with the teacher model to supervise the initial student model to perform distillation training to generate a final target student model. Therefore, the problems that in the prior art, model reasoning is delayed, the model deployment difficulty is large, the difference between model distillation training and reasoning is obvious, and the performance and efficiency of knowledge distillation are difficult to consider at the same time are solved.

The invention provides a data processing method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring to-be-processed data and a sparseness threshold set of a data processing model; inputting the to-be-processed data into the data processing model, and performing multi-round iterative processing on the to-be-processed data through the data processing model to obtain a first model output matrix; wherein during at least one round of iterative processing, a mask matrix is determined based on a first activation feature vector output by the activation layer and a sparsity threshold value, and the mask matrix is used for performing sparse calculation during the current step of iterative processing and multiple steps of iterative processing after the current step; and determining a data processing result of the to-be-processed data based on the first model output matrix. According to the invention, the calculation efficiency of the data processing model can be improved.

Embodiments are provided for converting a bit mask from a compressed format to an extended format. The bit mask is stored in a storage register in a compressed format. The compressed format does not include placeholder bits. The compressed format includes a single bit defining each element of a vector. The sizes of the elements can vary according to data types, for example, each element has 4, 8 and 16 bytes. In a single microoperation, the bitmask of the compressed format is converted to an extended format. The extended bit mask may be written to a predicate register. The extended bit mask includes a plurality of placeholder bits (the placeholder bits are not used in computation). The size of the extended bit mask corresponds to the size of the definition vector, including a plurality of elements of the data type. The extended bit mask may be used to execute an instruction. For example, a vector processor multiplies a sparse matrix with the vector using the extended bit mask.

The invention discloses an SIMT architecture-oriented program branch compilation optimization system, which relates to the technical field of program branch compilation optimization, and comprises a compiler rear end which takes code intermediate representation as input, executes hardware-related optimization and code generation work, converts the intermediate representation into binary codes and stores the binary codes in a video memory; the SIMT execution unit is responsible for reading the binary code from the video memory and executing a corresponding instruction; the SIMT stack is responsible for storing an address and an active mask of a non-jump branch when the Split instruction is executed, and is responsible for loading the stored non-executed branch and executing or setting the active mask to perform branch reconvergence after the Join instruction is executed; the branch performance analysis module is responsible for recording distribution of active threads of different branch paths; and the branch optimization module is responsible for analyzing and optimizing the code intermediate representation according to the recorded information, and replacing the input of the rear end of the compiler by using the optimized code intermediate representation. The code execution efficiency can be improved.

The invention discloses a hierarchical quantization and dynamic priority management-based model cache optimization method and device. The method comprises the following steps of: acquiring an importance score of each Token of a word sequence corresponding to a reasoning long text by utilizing a mask language model; constructing a Token importance scoring model, and dividing the cloud platform GPU into a head cache region, a middle cache region and a tail cache region; the service reasoning model stores the key value pairs of the first # imgabs0 # Token in a Token reasoning sequence corresponding to the reasoning long text in a head cache region in an FP16 format, stores the key value pairs of the first # imgabs1 # Token before the current processing Token in a tail cache region in the FP16 format according to an FIFO principle, and stores the key value pairs shifted out of the tail cache region in an intermediate cache region in an FP8 format; and after the intermediate cache region is full, updating the intermediate cache region by using the Token importance scoring model and the importance score. According to the method, the reasoning efficiency of the large language model in long text generation can be improved.

Systems and methods are provided that train a machine-learned language encoding model through the use of a contrastive learning task. In particular, the present disclosure describes a contrastive learning task where the encoder learns to distinguish input tokens from plausible alternatives. In some implementations, on each training example the proposed method masks out some subset (e.g., 15%) of the original input tokens, replaces the masked tokens with samples from a “generator” (e.g., which may be a small masked language model), and then trains the encoder to predict whether each token comes from the original data or is a replacement produced by the generator.

A decoding operation is performed by altering at least one bit of the bit string from a first value to a second value and applying a bit mask to each bit of the bit string that is not altered from the first value to the second value. The decoding operation further includes writing an indication corresponding to a quantity of bits that have been altered from the first value to the second value to an array of memory cells, wherein the indication corresponds to a quantity of errors contained in the bit string, determining that the quantity of errors for the bit string has reached a threshold quantity of errors, and refraining from performing a subsequent operation to alter at least the one bit of the bit string, or a different bit of the bit string, or both, from the first value to the second value in response to determining that the quantity of errors for the bit string has reached the threshold quantity of errors.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence that includes a respective token selected from a vocabulary of tokens at each of multiple output positions. In one aspect, one of the methods includes obtaining an initial output sequence, the initial output sequence comprising a mask token at each of at least a subset of the multiple output positions; repeatedly performing the following at each of multiple update iterations: obtaining an intermediate representation of the output sequence; generate a diffusion model output that comprises, for each of the multiple output positions, a respective score for each token in at least a subset of the vocabulary of tokens; determining, for each output position in the output sequence that is occupied by a mask token, a masked probability; selecting a subset of the multiple output positions; and generating an updated intermediate representation.

The embodiment of the invention provides an S-box circuit, and relates to the technical field of information security. The S-box circuit receives first data and mask data as input and outputs second data, and includes: a confusion adder that receives the first data and the mask data and outputs mask data; a first isomorphic mapping circuit that receives the masking data and outputs third data and fourth data; the first calculation circuit receives the third data, the fourth data and the fifth data and outputs sixth data; the inversion circuit is used for receiving sixth data and outputting seventh data; the second calculation circuit receives the third data, the fourth data and the seventh data and outputs eighth data; a third calculation circuit which receives the fourth data and the seventh data and outputs ninth data; and the second isomorphic mapping circuit receives the eighth data and the ninth data and outputs second data. The S-box circuit masks input data and output data through mask data, and has the capability of resisting power consumption analysis attack.

The invention relates to a large model fine tuning method, electronic equipment, a readable storage medium and a computer program product, and belongs to the field of deep learning. The method comprises the following steps: in the updating process of training, updating selected parameters through masks, and freezing unselected parameters; in the training process, the constraint of the L2 norm is added to the loss function. Compared with full-amount fine tuning, the large-model fine tuning method has the advantages that the training updating cost of the pre-training model is greatly reduced, and the large-model fine tuning method is suitable for multi-training-set and multi-parameter large-scale pre-training model fine tuning. The knowledge of the pre-training model is effectively migrated through data processing and regularization constraint of adding the L2 norm, the generalization ability is improved, the over-fitting condition is effectively solved, and the performance is improved. According to the method, no extra training parameters are added, and no time delay is added in the reasoning process. According to the invention, the calculation efficiency and fine tuning performance of fine tuning of the pre-training model are greatly improved.

Systems and methods related to efficient compression instruction handling in a processing pipeline are disclosed herein. A processing pipeline may accept a mask vector from a first register, the mask vector including a set of set bits, and may accept a payload from a second register. The processing pipeline may determine a set of prefix sums for a set of portions of the mask vector by applying the portions to a set of adders in parallel to add the set bits independently for each of the portions. The processing pipeline may determine a set of indexes for the set bits, with respect to the mask vector, in parallel, using the set of prefix sums and the set of portions of the mask vector. The processing pipeline may store a set of identified values from the payload, as identified by the set of indexes, in at least one destination register.

The invention discloses an automatic code defect repairing method based on a code attribute graph and a large model, which comprises the following steps: collecting a code sample, analyzing a source code, generating the code attribute graph fusing an abstract syntax tree, a control flow graph and a data flow graph, and then carrying out dynamic pruning according to node importance calculated based on node degrees, PageRank and data propagation dependency, so as to obtain a large model; key nodes are reserved to improve the graph learning efficiency; extracting features of the optimized graph through GAT, introducing multi-scale coding, and generating a graph embedding vector; in combination with node importance, high-value Tokens are screened, semantic embedding of structure perception is carried out, and more accurate text representation is obtained; generating a comprehensive feature by fusing the image and the text vector, and then inserting a mask at a position with weak dependence or high risk; performing fine tuning optimization on the mask position; and inputting the comprehensive features and the mask source codes into a pre-training language model to generate patches, performing multi-task evaluation and reordering, and selecting an optimal patch to complete repair.

The embodiment of the invention provides a processor core, a processor and a method for the processor. The processor core includes a processing pipeline configured to rename and execute a first instruction including at least one of a first architectural register and a second architectural register, and includes: a first type of physical register configured to be mapped by the first architectural register and configured to store a first type of data; the first type of physical register is configured to be mapped by the first architecture register and configured to store data of a first type, the second type of physical register is configured to be mapped by the second architecture register and configured to store data of a second type, the first architecture register comprises a vector architecture register, the data of the first type stored by the first type of physical register comprises vector data, and the second architecture register comprises a mask architecture register; the second type of data stored in the second type of physical register comprises mask data. The processor core may mitigate processing pipeline stagnation and increase less area.

The invention relates to the technical field of artificial intelligence, and relates to a large language model reasoning optimization method, system and device for resource-constrained equipment, and a medium. The method comprises the following steps: establishing a resource budget, acquiring video memory parameters of equipment, including a total available video memory and a current available video memory, and setting a memory total budget of a reasoning session; evaluating memory requirements, including static calculation and dynamic calculation; calculating a total memory demand, and if the total memory demand is greater than the total memory budget, starting an iterative optimization sub-process; and loading the configuration meeting the budget, and executing speculation decoding through the optimized attention mask, which comprises the following steps of: generating a candidate lexical element sequence in parallel based on a mask structure, verifying the candidate sequence in batches and accepting or rejecting lexical elements, and updating a confirmed sequence and key value cache. Through a collaborative optimization strategy, the reasoning speed, memory occupation and generation quality are intelligently balanced, so that efficient and stable large-model speculation decoding reasoning is realized on equipment with limited memory.

The invention relates to the technical field of artificial intelligence, in particular to an operation method and an arithmetic unit based on a Point data format.The method comprises the steps that sign bits, exponential expansion bits, mantissa bits and exponential bits of the Point data format are obtained, all the positions of the Point data format are preprocessed, a bit width distribution priority rule is defined, and a bit width distribution priority rule is obtained; multiplying the two mantissa bits obtained by decoding to obtain a mantissa product and a mask sequence; determining a prepositive zero number of the mantissa product according to the mask sequence, and performing unified logic shift on the mantissa product; the method comprises the following steps: carrying out shift on a mantissa part, carrying out rounding operation on the shifted mantissa part, converting a mantissa product into a fixed-point number format according to a sign bit, an exponential expansion bit and an exponential bit, carrying out additive operation based on the fixed-point number, and outputting an accumulation result. The circuit area and power consumption are reduced, complex time sequence control is not needed, and logic resource consumption is reduced.

Systems, apparatuses and methods may provide for technology that conducts, in accordance with a first instruction, a load of a block of data into a register, wherein the block of data is to include a plurality of lanes, conducts, in accordance with a second instruction, a first bitwise mask application to each lane in the plurality of lanes, and extracts a set of vector dimensions from the block of data based on the first bitwise mask application.

The invention provides a BERT model training method and system, computer equipment, a storage medium and a program product, and belongs to the technical field of natural language processing. The method comprises the following steps: respectively generating a plurality of sequences through a plurality of decoding paths by utilizing a pre-constructed BERT model; dividing the plurality of sequences into positive samples and negative samples; sampling the positive sample and the negative sample by using the reference model to construct a preference learning loss function; constructing a regular penalty term loss function; constructing a mask prediction loss function; and determining the total loss of the BERT model according to the preference learning loss function, the regular penalty term loss function and the mask prediction loss function to obtain a trained BERT model. By adopting the scheme, the BERT model is guided to learn the influence of different decoding paths on output, and the ability of the BERT model to select paths in a complex decoding space and generate sequences with higher quality is enhanced.

A system can comprise a memory that stores computer executable components, and a processor, operably coupled to the memory, that executes the computer executable components comprising: a linking component that associates one or more unmasked elements of the logical form with one or more corresponding structured knowledge elements of a knowledge base and a prediction component that predicts the one or more masked elements based on extended context of the corresponding structured knowledge elements of the knowledge base to generate one or more predicted elements. In an embodiment, the prediction component predicts the one or more masked elements based on scores of one or more candidate elements. In an embodiment, the system can determine one or more rules that describe the natural language text segment in terms of the structured knowledge elements and associated weights of the knowledge base paths.

The invention provides a verification code verification method and device, equipment, a medium and a program product, and aims to solve the problem that an automatic script in the related technology cannot realize automatic testing of verification of a verification code of a rotary sliding block. The method comprises the steps of performing separation processing on a rotating main body and a target pattern in a verification code image to obtain a main body binary mask corresponding to the rotating main body and a target binary mask corresponding to the target pattern; based on the main body binary mask and the target binary mask, a target angle difference between the rotating main body and the target pattern is calculated, and the target angle difference is used for indicating the rotating main body to rotate to a rotating angle matched with the target pattern; generating a rotation track based on the target angle difference by using a slow motion function, wherein the rotation track comprises random disturbance and a random pause point; and controlling the rotating main body to rotate based on the rotating track to obtain a verification result. According to the invention, automatic testing of verification of the rotary sliding block verification code can be realized.