Coding method and device for rotation position coding based on large language model

Abstract

The invention provides a large language model-based rotation position coding method, which comprises the following steps of: carrying out iterative calculation on a trigonometric function through a preset iterative calculation formula, and obtaining a current round of trigonometric function value through a previous round of trigonometric function value; and obtaining at least one input vector, and carrying out element-by-element multiplication and addition operation on the input vector and the trigonometric function value of the round to obtain a vector code of the rotation position code. The invention further provides a coding device of the rotation position coding based on the large language model, a storage medium and electronic equipment. Therefore, according to the method, the balance of calculation precision, calculation efficiency and hardware overhead is realized, and the on-chip storage requirement is remarkably reduced, so that powerful support is provided for acceleration of a large language model.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models of artificial intelligence, and in particular to a coding method, device, storage medium and electronic device based on rotational position coding of a large language model. Background Art

[0002] In recent years, large language models (LLMs) have demonstrated powerful performance in natural language processing, machine translation, and knowledge question answering. However, as the scale of model parameters increases, its computational complexity and storage requirements also increase dramatically, placing higher demands on hardware resources. In this context, how to optimize computing and storage efficiency while ensuring model performance has become a key technical challenge for accelerating large model reasoning.

[0003] Position encoding is a key component of large language models and is used to model position information in a sequence. Compared with traditional absolute position encoding, RoPE (Rotary Position Embedding) achieves accurate modeling of relative position relationships by dynamically adjusting the angular direction of the input feature vector. The introduction of rotational position encoding improves the expressiveness of large language models in long sequence tasks and is widely used in mainstream large language models. Although rotational position encoding has excellent performance, its computational complexity and resource requirements have also become important factors limiting hardware implementation. Therefore, exploring efficient rotational position encoding calculation methods has important research and application value.

[0004] In fact, there are challenges in hardware design for rotational position encoding. The hardware implementation of rotational position encoding needs to take into account the calculation accuracy, storage requirements and calculation efficiency under the condition of limited hardware resources. It specifically faces the following core challenges:

[0005] The first is the complexity of transcendental function calculation. The core of rotational position encoding is to rotate the eigenvector through sine and cosine functions. Although directly calculating these transcendental functions has high accuracy, the hardware overhead is huge and the hardware implementation cost is high, which is not suitable for forming an array to support the operation of rotational position encoding.

[0006] Second, the storage overhead is large. In order to reduce the computational complexity, the table lookup method is often used instead of direct calculation. However, whether it is a lookup table that directly stores the sine and cosine function values ​​or a linear interpolation table, it takes up a lot of storage resources. Therefore, although the table lookup method reduces the computational complexity, it sacrifices some accuracy and introduces a higher storage cost. It can be seen that the existing technology cannot maintain high-precision angle calculation while reducing storage overhead and computing resources.

[0007] In summary, the prior art obviously has inconveniences and defects in practical use, so it is necessary to improve it. Summary of the invention

[0008] In view of the above-mentioned defects, the purpose of the present invention is to provide a coding method, device, storage medium and electronic device based on rotational position coding of a large language model, which achieves a balance between computing accuracy, computing efficiency and hardware overhead, and significantly reduces on-chip storage requirements, thereby providing strong support for the acceleration of large language models.

[0009] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0010] In a first aspect, an embodiment of the present invention provides a method for encoding rotational position encoding, the method comprising:

[0011] Iteratively calculate the trigonometric function through a predetermined iterative calculation formula, and obtain the trigonometric function value of this round through the trigonometric function value of the previous round;

[0012] At least one input vector is obtained, and element-by-element multiplication and addition operations are performed on the input vector and the trigonometric function value of this round to obtain a vector code of the rotation position code.

[0013] According to the encoding method of rotational position encoding based on a large language model of the present invention, the iterative calculation formula is:

[0014]

[0015] Said represents the universal quantifier; m represents the number of iterations; i is a variable used to index the input vector, and the value range of i is a natural number from 0 to 31; t represents the incremental angle; sin represents the sine function, cos represents the cosine function, sin(m)t i and cos(m)t i Indicates the value of the previous round of trigonometric functions, which is the input data for this round of calculation; sin(m+1)t i and cos(m+1)t i Represents the trigonometric function value of this round.

[0016] According to the encoding method of rotational position encoding based on a large language model of the present invention, the iterative calculation formula uses a sum angle formula to iteratively calculate the trigonometric function, and the sum angle formula is:

[0017] sin(A+B)=sinA*cosB+cosA*sinB; and

[0018] cos(A+B)=cosA*cosB-sinA*sinB.

[0019] According to the encoding method of rotational position encoding based on a large language model of the present invention, the step of obtaining the trigonometric function value of the current round by using the trigonometric function value of the previous round includes:

[0020] Storing the trigonometric function values ​​of this round;

[0021] Delete the previous round of trigonometric function values.

[0022] The encoding method based on rotational position encoding of a large language model according to the present invention further includes:

[0023] Splitting the encoding calculation process of the rotational position encoding into at least two pipeline stages;

[0024] In the first stage, the trigonometric function is iteratively calculated by the iterative calculation formula, and the trigonometric function value of the current round is obtained by the trigonometric function value of the previous round;

[0025] In the second stage, the trigonometric function value of this round is multiplied and added element by element with the input vector to obtain the vector code of the rotation position code.

[0026] According to the encoding method of rotational position encoding based on a large language model of the present invention, the step of performing element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round includes:

[0027] The input vector and the trigonometric function value of this round are subjected to element-by-element multiplication and addition operations through a predetermined multiplication-addition coding array.

[0028] According to the encoding method of rotational position encoding based on a large language model of the present invention, the step of obtaining at least one input vector and performing element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round includes:

[0029] A group of the input vectors is obtained, and an element-by-element multiplication and addition operation is performed on the group of the input vectors and the trigonometric function values ​​of this round.

[0030] In a second aspect, an embodiment of the present invention provides an encoding device for rotational position encoding based on a large language model constructed based on any of the methods described above, the device comprising:

[0031] An iterative calculation module performs iterative calculations on trigonometric functions through a predetermined iterative calculation formula, and obtains the trigonometric function values ​​of this round through the trigonometric function values ​​of the previous round;

[0032] The vector encoding module obtains at least one input vector, performs element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round, and obtains a vector code of the rotation position code.

[0033] In a third aspect, an embodiment of the present invention provides a storage medium for storing a computer program for executing any of the above-mentioned methods for rotating position coding based on a large language model.

[0034] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a storage medium, a processor, and a computer program stored in the storage and executable on the processor, wherein the processor implements the encoding method for rotational position encoding based on a large language model according to any one of claims 1 to 7 when executing the computer program.

[0035] The encoding method of the rotational position encoding of the present invention includes: an iterative calculation step, iteratively calculating the trigonometric function through an iterative calculation formula, and obtaining the trigonometric function value of this round through the trigonometric function value of the previous round; a vector encoding step, obtaining at least one input vector, performing element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round, and obtaining the vector encoding of the rotational position encoding. Thereby, the present invention can encode the input vector by iteratively calculating the trigonometric function value, avoiding directly calculating the transcendental function or storing a large-capacity lookup table, and can take into account the calculation accuracy, calculation efficiency and hardware overhead, and achieve a balance between the calculation accuracy, calculation efficiency and hardware overhead; and the present invention only needs to store the trigonometric function value of the previous round, without pre-storing a large-scale lookup table or interpolation data, significantly reducing the on-chip storage requirements, thereby providing strong support for the acceleration of large language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flowchart of a coding method based on rotational position coding of a large language model provided in the first embodiment of the present invention;

[0037] Figure 2 is a flow chart of a pipeline encoding method for rotational position encoding provided in the second embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of a trigonometric function iterative calculation unit provided in Embodiment 3 of the present invention;

[0039] Figure 4 is a schematic diagram of a multiplication-addition coding array provided in Embodiment 3 of the present invention;

[0040] Figure 5 Schematic diagram of pipeline coding of rotational position coding provided by Embodiment 3 of the present invention;

[0041] Figure 6 is a structural schematic diagram of an encoding device for rotational position encoding based on a large language model provided in Embodiment 1 of the present invention;

[0042] Figure 7is a structural diagram of an encoding device for rotational position encoding based on a large language model provided in Embodiment 2 of the present invention;

[0043] Figure 8 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0045] It should be noted that references to "one embodiment", "embodiment", "example embodiment", etc. in this specification refer to the embodiment described, which may include specific features, structures or characteristics, but not every embodiment must include these specific features, structures or characteristics. In addition, such expressions do not refer to the same embodiment. Furthermore, when describing specific features, structures or characteristics in conjunction with an embodiment, whether or not there is an explicit description, it has been shown that it is within the knowledge of those skilled in the art to combine such features, structures or characteristics into other embodiments.

[0046] In addition, certain words are used in the specification and subsequent claims to refer to specific components or parts. Those with ordinary knowledge in the relevant field should understand that manufacturers can use different nouns or terms to refer to the same component or part. This specification and subsequent claims do not use differences in names as a way to distinguish components or parts, but use differences in the functions of components or parts as the criteria for distinction. "Including" and "including" mentioned throughout the specification and subsequent claims are open-ended terms and should be interpreted as "including but not limited to". In addition, the word "connected" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connection through other devices.

[0047] In the following, in conjunction with the accompanying drawings, the encoding method of rotational position encoding based on a large language model provided by the embodiment of the present invention is described in detail through specific embodiments and application scenarios.

[0048] Figure 1 Yes, including the steps:

[0049] This invention focuses on the efficient implementation of rotational position encoding, aiming to design a computing device that can balance computational accuracy and hardware efficiency, and provide support for the acceleration of large models. After studying the law of angle iteration, it is found that the input vector can be encoded by iteratively calculating the trigonometric function value through the sum angle formula, avoiding the direct calculation of transcendental functions or the storage of large-capacity lookup tables, and achieving a balance between computational accuracy and hardware overhead.

[0050] Figure 1 : is a flow chart of a coding method based on rotational position coding of a large language model provided in Embodiment 1 of the present invention, the method comprising:

[0051] Step S101, iteratively calculating the trigonometric function by using a predetermined iterative calculation formula, and obtaining the trigonometric function value of this round by using the trigonometric function value of the previous round.

[0052] Preferably, the iterative calculation formula is:

[0053]

[0054] represents the universal quantifier; m represents the number of iterations; i is a variable used to index the input vector, and the value range of i is a natural number from 0 to 31; t represents the incremental angle; sin represents the sine function, cos represents the cosine function, sin(m)t i and cos(m)t i Indicates the value of the previous round of trigonometric functions, which is the input data for this round of calculation; sin(m+1)t i and cos(m+1)t i Represents the trigonometric function value of this round.

[0055] Preferably, if Figure 3 As shown, the iterative calculation formula uses the sum angle formula to iteratively calculate the trigonometric function, and the sum angle formula is:

[0056] sin(A+B)=sinA*cosB+cosA*sinB; and

[0057] cos(A+B)=cosA*cosB-sinA*sinB.

[0058] The present invention only needs to store a set of historical trigonometric function values ​​and constant values. The unit multiplies the input sin mt and cos mt by the stored fixed values ​​and then adds them to obtain sin(m+1)t and cos(m+1)t, and then it can continue the iterative calculation.

[0059] The new trigonometric function value required for the rotation position encoding of the present invention can be obtained by performing a simple multiplication and addition operation on the trigonometric function value of the previous round.

[0060] Step S102, obtaining at least one input vector, performing element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round, and obtaining a vector code of the rotation position code.

[0061] Preferably, if Figure 4 As shown, a group of input vectors are obtained, and a group of input vectors and the trigonometric function values ​​of this round are multiplied and added element by element, that is, a group of input vectors share a set of trigonometric function values ​​for encoding. According to the algorithm implementation of rotation position encoding, the trigonometric function values ​​and the input vectors are multiplied and added element by element to complete vector encoding.

[0062] The encoding method of the rotation position encoding of the present invention proposes an implementation scheme of the rotation position encoding calculation component based on the iteration of the sum angle formula, which can avoid high computational overhead and reduce the on-chip storage space occupation on the basis of ensuring high precision. It has the following technical effects:

[0063] (1) Balance between computational accuracy and hardware efficiency. The present invention uses the sum angle formula for iterative calculation, which not only avoids the high complexity calculation of transcendental functions, but also reduces the resource consumption of table lookup and storage, thus achieving a comprehensive balance between computational accuracy, storage and computational efficiency.

[0064] (2) Optimization of hardware area and timing performance. The present invention significantly reduces the complexity of a single-stage computing unit by splitting the rotational position encoding calculation into multiple pipeline stages. At the same time, the pipeline hierarchical design effectively reduces the critical path delay, increases the clock frequency of hardware operation, and optimizes the overall timing performance. Compared with the existing technology, it has higher hardware utilization efficiency.

[0065] Figure 2 : is a flow chart of a pipeline encoding method for rotational position encoding provided by Embodiment 2 of the present invention, the method comprising:

[0066] Step S201, iteratively calculating the trigonometric function by using a predetermined iterative calculation formula, and obtaining the trigonometric function value of this round by using the trigonometric function value of the previous round.

[0067] Preferably, the iterative calculation formula is:

[0068]

[0069] represents the universal quantifier; m represents the number of iterations; i is a variable used to index the input vector, and the value range of i is a natural number from 0 to 31; t represents the incremental angle; sin represents the sine function, cos represents the cosine function, sin(m)t i and cos(m)t i Indicates the value of the previous round of trigonometric functions, which is the input data for this round of calculation; sin(m+1)ti and cos(m+1)t i Represents the trigonometric function value of this round.

[0070] Preferably, if Figure 3 As shown, the iterative calculation formula uses the sum angle formula to iteratively calculate the trigonometric function, and the sum angle formula is:

[0071] sin(A+B)=sinA*cosB+cosA*sinB; and

[0072] cos(A+B)=cosA*cosB-sinA*sinB.

[0073] The present invention only needs to store a set of historical trigonometric function values ​​and constant values. The unit multiplies the input sin mt and cos mt by the stored fixed values ​​and then adds them to obtain sin(m+1)t and cos(m+1)t, and then it can continue the iterative calculation.

[0074] Step S202, storing the trigonometric function values ​​of this round and deleting the trigonometric function values ​​of the previous round.

[0075] Step S203, obtaining at least one input vector, performing element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round, and obtaining a vector code of the rotation position code.

[0076] Preferably, if Figure 4 As shown, a group of input vectors are obtained, and a group of input vectors and the trigonometric function values ​​of this round are multiplied and added element by element, that is, a group of input vectors share a set of trigonometric function values ​​for encoding. According to the algorithm implementation of rotation position encoding, the trigonometric function values ​​and the input vectors are multiplied and added element by element to complete vector encoding.

[0077] Preferably, if Figure 4 The schematic diagram of the multiplication-addition coding array shown in the figure performs element-by-element multiplication-addition operations on the input vector and the trigonometric function value of this round through the predetermined multiplication-addition coding array. The present invention adopts a high-throughput multiplication-addition array, which has a reasonable organizational structure and can complete the rotation position encoding of the entire vector with only a single input, without the need for slow input processing in steps, and has high-throughput computing capability.

[0078] In the encoding method of the rotation position encoding based on the large language model of the present invention, the pipeline encoding of the rotation position encoding is adopted, such as Figure 5 As shown, specifically including:

[0079] Splitting the encoding calculation process of the rotational position encoding into at least two pipeline stages;

[0080] In the first stage, the trigonometric functions are iteratively calculated by using the iterative calculation formula, and the trigonometric function values ​​of this round are obtained by using the trigonometric function values ​​of the previous round;

[0081] In the second stage, the trigonometric function value of this round is multiplied and added element by element with the input vector to obtain the vector code of the rotation position code.

[0082] The present invention reasonably divides the calculation process of the rotation position encoding into multiple pipeline stages, which can significantly optimize the hardware area and timing performance while maintaining high throughput.

[0083] The present invention iteratively calculates trigonometric function values ​​through the sum angle formula, which has the following technical effects:

[0084] (1) Simple operation unit: Instead of directly calculating trigonometric functions each time, the algorithm only needs multiplication and addition operations, thus avoiding the high computational cost of transcendental function operations, greatly improving the computing efficiency of the hardware, and reducing the reliance on complex mathematical operation units.

[0085] (2) Small storage area: Only fixed incremental angles and the trigonometric function values ​​of the previous round need to be stored, without the need to pre-store large-scale lookup tables or interpolation data, which significantly reduces on-chip storage requirements.

[0086] (3) High calculation accuracy: Compared with the interpolation table lookup method, iterative calculation uses a fixed mathematical formula to maintain high-precision calculation and avoids the accumulation of interpolation errors.

[0087] It should be noted that the encoding method based on rotational position encoding of a large language model provided in the embodiment of the present invention can be executed by an electronic device, an encoding device based on rotational position encoding of a large language model, or a control module in the encoding device based on rotational position encoding of a large language model for executing the encoding method based on rotational position encoding of a large language model. In the embodiment of the present invention, an encoding device based on rotational position encoding of a large language model executes the encoding method based on rotational position encoding of a large language model as an example to illustrate the encoding device based on rotational position encoding of a large language model provided in the embodiment of the present invention.

[0088] Figure 6 1 is a schematic diagram of the structure of an encoding device for rotational position encoding based on a large language model provided in Embodiment 1 of the present invention. The encoding device 100 comprises at least an iterative calculation module 10 and a vector encoding module 20, wherein:

[0089] The iterative calculation module 10 performs iterative calculation on the trigonometric function using a predetermined iterative calculation formula, and obtains the trigonometric function value of this round using the trigonometric function value of the previous round.

[0090] The vector encoding module 20 obtains at least one input vector, performs element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round, and obtains a vector code of the rotation position code.

[0091] Figure 7 1 is a schematic diagram of the structure of an encoding device for rotational position encoding based on a large language model provided in Embodiment 2 of the present invention. The encoding device 100 at least includes an iterative calculation module 10 and a vector encoding module 20, wherein:

[0092] The iterative calculation module 10 performs iterative calculation on the trigonometric function using a predetermined iterative calculation formula, and obtains the trigonometric function value of this round using the trigonometric function value of the previous round.

[0093] Preferably, the iterative calculation formula is:

[0094]

[0095] Said represents the universal quantifier; m represents the number of iterations; i is a variable used to index the input vector, and the value range of i is a natural number from 0 to 31; t represents the incremental angle; sin represents the sine function, cos represents the cosine function, sin(m)t i and cos(m)t i Indicates the value of the previous round of trigonometric functions, which is the input data for this round of calculation; sin(m+1)t i and cos(m+1)t i Represents the trigonometric function value of this round.

[0096] Preferably, the iterative calculation formula uses a sum angle formula to iteratively calculate the trigonometric function, and the sum angle formula is:

[0097] sin(A+B)=sinA*cosB+cosA*sinB; and

[0098] cos(A+B)=cosA*cosB-sinA*sinB.

[0099] The vector encoding module 20 obtains at least one input vector, performs element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round, and obtains a vector code of the rotation position code.

[0100] Preferably, the vector encoding module 20 is used to perform element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round through a predetermined multiplication and addition encoding array.

[0101] Preferably, the vector encoding module 20 is used to obtain a group of the input vectors, and perform element-by-element multiplication and addition operations on the group of input vectors and the trigonometric function values ​​of this round.

[0102] Preferably, the encoding device 100 for rotational position encoding based on a large language model of the present invention further includes at least a storage processing module 30 for storing the current round of trigonometric function values ​​and deleting the previous round of trigonometric function values ​​after obtaining the current round of trigonometric function values ​​through the previous round of trigonometric function values.

[0103] Preferably, the encoding device 100 of the rotational position encoding based on the large language model of the present invention also includes at least a pipeline classification module 40, which is used to split the encoding calculation process of the rotational position encoding into at least two pipeline stages; in the first stage, the trigonometric function is iteratively calculated by the iterative calculation formula, and the trigonometric function value of the current round is obtained by the trigonometric function value of the previous round; in the second stage, the trigonometric function value of the current round is multiplied and added element by element with the input vector to obtain the vector encoding of the rotational position encoding.

[0104] The encoding device based on rotation position encoding of a large language model provided by the embodiment of the present invention can achieve Figures 1 to 5 To avoid repetition, the various processes implemented in the embodiment of the encoding method based on rotational position encoding of a large language model are not described here.

[0105] The encoding device for rotational position encoding based on a large language model provided in an embodiment of the present invention includes: an iterative calculation module, which iteratively calculates trigonometric functions through iterative calculation formulas, and obtains the current round of trigonometric function values ​​through the previous round of trigonometric function values; a vector encoding module, which obtains at least one input vector, performs element-by-element multiplication and addition operations on the input vector and the current round of trigonometric function values, and obtains the vector encoding of the rotational position encoding. Thereby, the present invention can encode the input vector by iteratively calculating the trigonometric function values, avoids directly calculating transcendental functions or storing large-capacity lookup tables, can take into account the calculation accuracy, calculation efficiency and hardware overhead, and achieves a balance between the calculation accuracy, calculation efficiency and hardware overhead; and the present invention only needs to store the previous round of trigonometric function values, and does not need to pre-store large-scale lookup tables or interpolation data, which significantly reduces the on-chip storage requirements, thereby providing strong support for the acceleration of large language models.

[0106] The present invention also provides a storage medium for storing Figure 1 to Figure 5A computer program of any of the encoding methods of the rotational position encoding based on the large language model. For example, a computer program instruction, when it is executed by a computer, through the operation of the computer, can call or provide the method and / or technical solution according to the present invention, and can achieve the same technical effect. To avoid repetition, it will not be repeated here. The program instruction for calling the method of the present invention may be stored in a fixed or removable storage medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium and / or stored in a storage medium of a computer device that runs according to the program instruction.

[0107] According to one embodiment of the present invention, the present invention also provides a Figure 8 The electronic device 400 shown, the electronic device 400 optionally includes a storage medium 200 for storing a computer program and a processor 300 for executing the computer program, wherein when the computer program is executed by the processor 300, any of the above-mentioned encoding methods based on the rotational position encoding of the large language model is implemented, triggering the electronic device 400 to execute the method and / or technical solution based on the aforementioned multiple embodiments, and can achieve the same technical effect, to avoid repetition, it is not repeated here. It should be noted that the electronic device in the embodiment of the present invention includes a mobile electronic device and a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, a super mobile personal computer, a netbook or a personal digital assistant, etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present invention.

[0108] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present invention (including related data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and the like. In addition, some steps or functions of the present invention can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0109] The present invention can be implemented on a computer as a computer-implemented method, or implemented in dedicated hardware, or implemented in a combination of the two. The executable code or part thereof for the method according to the present invention can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Optionally, the computer program product includes a non-temporary program code component stored on a computer-readable medium so as to perform the method according to the present invention when the program product is executed on a computer.

[0110] In an alternative embodiment, the computer program comprises computer program code means adapted to perform all the steps of the method according to the invention when the computer program is run on a computer. Optionally, the computer program is embodied on a computer readable medium.

[0111] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0112] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A coding method based on rotational position coding of a large language model, characterized in that: include: Iteratively calculate the trigonometric function through a predetermined iterative calculation formula, and obtain the trigonometric function value of this round through the trigonometric function value of the previous round; At least one input vector is obtained, and element-by-element multiplication and addition operations are performed on the input vector and the trigonometric function value of this round to obtain a vector code of the rotation position code.

2. The encoding method of rotational position encoding based on a large language model according to claim 1, characterized in that: The iterative calculation formula is: Said represents the universal quantifier; m represents the number of iterations; i is a variable used to index the input vector, and the value range of i is a natural number from 0 to 31; t represents the incremental angle; sin represents the sine function, cos represents the cosine function, sin(m)t i and cos(m)t i Indicates the value of the previous round of trigonometric functions, which is the input data for this round of calculation; sin(m+1)t i and cos(m+1)t i Represents the trigonometric function value of this round.

3. The encoding method based on rotational position encoding of a large language model according to claim 2, characterized in that: The iterative calculation formula uses the sum angle formula to iteratively calculate the trigonometric function, and the sum angle formula is: sin(A+B)=sinA*cosB+cosA*sinB; and cos(A+B)=cosA*cosB-sinA*sinB.

4. The encoding method based on rotational position encoding of a large language model according to claim 1, characterized in that: The step of obtaining the trigonometric function value of this round by using the trigonometric function value of the previous round includes: Storing the trigonometric function values ​​of this round; Delete the previous round of trigonometric function values.

5. The encoding method of rotational position encoding based on a large language model according to claim 1, characterized in that: Also includes: Splitting the encoding calculation process of the rotational position encoding into at least two pipeline stages; In the first stage, the trigonometric function is iteratively calculated by the iterative calculation formula, and the trigonometric function value of the current round is obtained by the trigonometric function value of the previous round; In the second stage, the trigonometric function value of this round is multiplied and added element by element with the input vector to obtain the vector code of the rotation position code.

6. The encoding method based on rotational position encoding of a large language model according to claim 1, characterized in that: The step of performing element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round comprises: The input vector and the trigonometric function value of this round are subjected to element-by-element multiplication and addition operations through a predetermined multiplication-addition coding array.

7. The encoding method based on rotational position encoding of a large language model according to claim 1, characterized in that: The step of obtaining at least one input vector and performing element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round comprises: A group of the input vectors is obtained, and an element-by-element multiplication and addition operation is performed on the group of the input vectors and the trigonometric function values ​​of this round.

8. An encoding device based on rotational position encoding of a large language model constructed based on the method according to any one of claims 1 to 7, characterized in that: include: An iterative calculation module performs iterative calculations on trigonometric functions through a predetermined iterative calculation formula, and obtains the trigonometric function values ​​of this round through the trigonometric function values ​​of the previous round; The vector encoding module obtains at least one input vector, performs element-by-element multiplication and addition operations on the input vector and the trigonometric function value of this round, and obtains a vector code of the rotation position code.

9. A storage medium, characterized in that: A computer program for storing a computer program for executing the encoding method based on rotational position encoding of a large language model according to any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the encoding method based on rotational position encoding of any one of claims 1 to 7 is implemented.

Images