Model coding, model reasoning method, device, electronic device and storage medium

Abstract

The present invention provides a model encoding, a model reasoning method, a device, an electronic device and a storage medium, wherein the model encoding method comprises: determining a current input word segmentation; inputting the current input word segmentation into a model, and the model performs a table lookup in a corresponding input word table based on a multidimensional word segmentation identifier corresponding to the current input word segmentation, obtains a table lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains an input word segmentation vector based on each table lookup input word segmentation vector. The model encoding, model reasoning method, device, electronic device and storage medium provided by the present invention use a multidimensional word segmentation identifier to represent the current input word segmentation, that is, use multiple token ids to represent a token, thereby reducing the word table size and saving memory and computing overhead.

Description

Technical Field

[0001] The present invention relates to the field of model reasoning technology, and in particular to a model encoding, a model reasoning method, a device, an electronic device and a storage medium. Background Art

[0002] In the reasoning process of large text models, it is usually necessary to encode each token and look up the corresponding token vector to support reasoning calculations. However, as the model size increases and the need to support more tokens increases, related technologies often use larger vocabularies to represent more tokens, but this results in huge memory consumption.

[0003] For example, for a model that supports 125k tokens (such as the baichuan2_7B model), the vocabulary size that needs to be stored is 125K×4K×4×2=4GB, while the space required to store the model weights is 14GB. That is, the space occupied by the vocabulary is close to the space occupied by the model weights, increasing the storage and computing overhead. Summary of the invention

[0004] The present invention provides a model encoding, a model reasoning method, a device, an electronic device and a storage medium, which are used to solve the defect that the vocabulary table of the model in the prior art occupies a large space, thereby increasing the storage and computing overhead.

[0005] The present invention provides a model encoding method, comprising:

[0006] Determine the current input word segmentation;

[0007] The current input word segmentation is input into the model, and the model performs a lookup in the corresponding input word table based on the multi-dimensional word segmentation identifier corresponding to the current input word segmentation, obtains the lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains the input word segmentation vector based on each lookup input word segmentation vector.

[0008] According to a model encoding method provided by the present invention, the step of obtaining an input word segmentation vector based on each table lookup input word segmentation vector includes:

[0009] Combine the word segmentation vectors of each lookup table input to obtain a combined word segmentation vector;

[0010] Based on the combined word segmentation vector and the corresponding weight matrix, the input word segmentation vector is determined.

[0011] The present invention also provides a model reasoning method, comprising:

[0012] Based on the model encoding method described above, determine the input word segmentation vector;

[0013] Model inference is performed based on the input word segmentation vector to obtain an inference result.

[0014] According to a model inference method provided by the present invention, the model inference is performed based on the input word segmentation vector to obtain an inference result, including:

[0015] Perform model inference based on the input word segmentation vector to obtain an output word segmentation vector;

[0016] The inference result is determined based on the output vocabulary matrix and the output word segmentation vector.

[0017] According to a model reasoning method provided by the present invention, the determining of the reasoning result based on the output vocabulary matrix and the output word segmentation vector includes:

[0018] The output word segmentation vector is multiplied by the output word list matrix, and the inference result is determined based on the multiplication result.

[0019] The present invention also provides a model encoding device, comprising:

[0020] A first determination unit, used to determine the current input word segmentation;

[0021] The model encoding unit is used to input the current input word segmentation into the model, and the model performs a table lookup in the corresponding input word table based on the multi-dimensional word segmentation identifier corresponding to the current input word segmentation, obtains the table lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains the input word segmentation vector based on each table lookup input word segmentation vector.

[0022] The present invention also provides a model reasoning device, comprising:

[0023] A second determining unit, used to determine an input word segmentation vector based on the model encoding method as described above;

[0024] The model inference unit is used to perform model inference based on the input word segmentation vector to obtain an inference result.

[0025] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements any one of the above-described model encoding methods or any one of the above-described model reasoning methods.

[0026] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program implements any of the model encoding methods described above, or implements any of the model reasoning methods described above.

[0027] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the model encoding methods described above, or implements any one of the model reasoning methods described above.

[0028] The model encoding, model inference method, device, electronic device and storage medium provided by the present invention use a multi-dimensional word segmentation identifier to represent the current input word segmentation, that is, use multiple token IDs to represent a token, thereby reducing the size of the vocabulary and saving memory and computing overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 It is a flow chart of the model encoding method provided by the present invention;

[0031] Figure 2 is a schematic diagram of encoding of the current input word segmentation provided by the present invention;

[0032] Figure 3 It is a flow chart of the model reasoning method provided by the present invention;

[0033] Figure 4 It is a decoding schematic diagram of the output word segmentation provided by the present invention;

[0034] Figure 5 It is a structural schematic diagram of the model encoding device provided by the present invention;

[0035] Figure 6 It is a structural schematic diagram of the model reasoning device provided by the present invention;

[0036] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In the reasoning process of large text models, each reasoning generates a token, and after multiple rounds of reasoning, a complete sentence or paragraph is obtained.

[0039] When a large model is inferred, each token is represented by a token id. At the beginning of inference, a table lookup is performed based on the token id to obtain the input token vector for inference calculation. Taking the llama7B model as an example, the vocabulary can represent up to 32k tokens. A table lookup is performed in the input vocabulary based on the token id to obtain an input token vector of size 1×4096 (here 4096 is a hidden dimension) for subsequent inference to obtain an output token vector. Among them, the input vocabulary size is 32k×4k×4=512M in the fp32 data format.

[0040] When the output token vector is obtained, a matrix multiplication operation is performed on the output token vector and the output vocabulary matrix, and the corresponding output token id is obtained after the logits result is sampled. Taking llama7B as an example, the corresponding output vocabulary size is 4k×32k×4=512M.

[0041] As can be seen from the above, in the related art, tokens before and after inference are represented by a corresponding token id. The more tokens that need to be represented, the larger the space occupied by the corresponding vocabulary. For example, for a model with hidden-sim=4096, if 32k tokens need to be represented, the corresponding vocabulary size in the fp32 data format is at least 32k×4k×4(fp32); if 128k tokens need to be represented, the corresponding vocabulary size in the fp32 data format is at least 128k×4k×4(fp32).

[0042] In addition, the larger the space occupied by the vocabulary, the smaller the number of parallel tasks (number of batches) that can be inferred, which in turn affects the model reasoning performance.

[0043] To this end, the present invention provides a model encoding method. Figure 1 It is a flow chart of the model encoding method provided by the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0044] Step 110, determine the current input word segmentation;

[0045] Step 120: input the current input word segmentation into the model, and the model performs a lookup in the corresponding input word table based on the multi-dimensional word segmentation identifier corresponding to the current input word segmentation, obtains the lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains the input word segmentation vector based on each lookup input word segmentation vector.

[0046] Here, the current input token can be understood as the input token of the current reasoning stage. For example, the current input token may be "I", and the next token of "I" needs to be inferred in the current reasoning stage.

[0047] In addition, at the beginning of the current reasoning phase, the current input word segmentation needs to be encoded to obtain the corresponding input word segmentation vector for subsequent model reasoning.

[0048] In addition, the multi-dimensional word segmentation identifier can be understood as the identifier of the current input word segmentation in multiple input word lists, that is, the current input word segmentation is represented by a multi-dimensional token id. Among them, the current input word segmentation can be represented by 2 word segmentation identifiers, 3 word segmentation identifiers, or n word segmentation identifiers, which is not specifically limited in the embodiment of the present invention.

[0049] Taking the current input word segmentation represented by two word segmentation identifiers as an example, if the multidimensional word segmentation identifier of the current input word segmentation is (82,215), it can be interpreted as the current input word segmentation is in the 82nd row of input word table 1 and in the 215th row of input word table 2.

[0050] Based on the multi-dimensional word segmentation identifier corresponding to the current input word segmentation, a table lookup is performed in the corresponding input word table to obtain the table lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and then the input word segmentation vector is obtained based on each table lookup input word segmentation vector. Optionally, each table lookup input word segmentation vector can be combined to obtain a combined word segmentation vector, and the combined word segmentation vector and the corresponding weight matrix are multiplied to obtain the input word segmentation vector, that is, the encoding of the current input word segmentation is completed for subsequent model reasoning. Among them, the input word table refers to the word table corresponding to each word segmentation, and the dimension of the word segmentation identifier corresponds to the number of input word tables. For example, when the current input word segmentation is represented by 2 word segmentation identifiers, it corresponds to 2 input word tables; when the current input word segmentation is represented by 3 word segmentation identifiers, it corresponds to 3 input word tables; when the current input word segmentation is represented by n word segmentation identifiers, it corresponds to n input word tables.

[0051] The embodiment of the present invention adopts a multi-dimensional word segmentation identifier to represent the current input word segmentation, that is, multiple token IDs are used to represent one token, which can reduce the size of the word list and save memory and computing overhead.

[0052] For example, assuming that the number of tokens supported by the model is 640k, and the hidden dimension (hidden_dim) is 4k, if one token id is used to represent one token, it corresponds to a larger vocabulary. In the fp32 data format, the vocabulary size is 640k×4k×4=10GB. If two token ids are used to represent one token, it corresponds to two vocabulary. In the fp32 data format, the sizes of the two vocabulary are both 0.8k×4k×4=12.8MB, and the total space occupied by the two vocabulary is 12.8MB×2=25.6MB, which is much smaller than the space occupied by the above vocabulary (10GB), and also saves computing overhead.

[0053] As an optional embodiment, the model here can be a text generation model, and the current input word segmentation can be determined based on the input text. The model encoding method of the above embodiment is used to obtain an input word segmentation vector, and the input word segmentation vector is used for subsequent model reasoning to obtain output text.

[0054] The model encoding method provided by the embodiment of the present invention adopts a multi-dimensional word segmentation identifier to represent the current input word segmentation, that is, multiple token ids are used to represent one token, thereby reducing the size of the vocabulary and saving memory and computing overhead.

[0055] Based on the above embodiment, the input word segmentation vector is obtained based on each table lookup input word segmentation vector, including:

[0056] Combine the word segmentation vectors of each lookup table input to obtain a combined word segmentation vector;

[0057] Based on the combined word segmentation vector and the corresponding weight matrix, the input word segmentation vector is determined.

[0058] Specifically, each lookup input word segmentation vector is found in the corresponding input word table based on each dimension word segmentation identifier of the current input word segmentation. After obtaining the lookup input word segmentation vector, each lookup input word segmentation vector is combined (eg, concatenated) to obtain a combined word segmentation vector.

[0059] Next, based on the combined word segmentation vector and the corresponding weight matrix, an input word segmentation vector is determined. For example, the combined word segmentation vector and the corresponding weight matrix are multiplied to obtain the input word segmentation vector.

[0060] Figure 2 is a schematic diagram of the encoding of the current input word segmentation provided by the present invention, such as Figure 2As shown, the current input word segmentation is represented by two word segmentation identifiers, token id1=82 and token id2=215. Based on token id1=82, the table is looked up in the input word table 1 to determine the table lookup input word segmentation vector 1. Based on token id2=215, the table is looked up in the input word table 2 to determine the table lookup input word segmentation vector 2. The table lookup input word segmentation vector 1 and the table lookup input word segmentation vector 2 are combined to obtain a combined word segmentation vector, and the combined word segmentation vector is multiplied by the corresponding weight matrix to obtain the input word segmentation vector.

[0061] Based on any of the above embodiments, Figure 3 It is a flow chart of the model reasoning method provided by the present invention, such as Figure 3 As shown, the method includes:

[0062] Step 310: Determine an input word segmentation vector based on the model encoding method described in any of the above embodiments;

[0063] Step 320: Perform model inference based on the input word segmentation vector to obtain an inference result.

[0064] Specifically, according to the aforementioned model encoding method, the current input word segmentation is encoded to determine the input word segmentation vector corresponding to the current input word segmentation. After the input word segmentation vector is obtained, model inference is performed based on the input word segmentation vector to obtain an inference result.

[0065] Based on any of the above embodiments, model inference is performed based on the input word segmentation vector to obtain inference results, including:

[0066] Perform model inference based on the input word segmentation vector to obtain the output word segmentation vector;

[0067] The inference result is determined based on the output vocabulary matrix and the output word segmentation vector.

[0068] Specifically, in the current reasoning stage, the model is reasoned based on the input word segmentation vector to obtain the output word segmentation vector. At the end of the current reasoning stage, the output word segmentation vector needs to be decoded to obtain the reasoning result.

[0069] As an optional embodiment, the inference result is determined based on the output vocabulary matrix and the output word segmentation vector. The output vocabulary matrix refers to the matrix corresponding to the output vocabulary, and the number of output vocabulary corresponds to the number of input vocabulary in the above text. For example, if the number of input vocabulary is 2, the number of output vocabulary is also 2.

[0070] Based on any of the above embodiments, based on the output vocabulary matrix and the output word segmentation vector, determining the inference result includes:

[0071] The output word segmentation vector is multiplied by the output word list matrix, and the inference result is determined based on the multiplication result.

[0072] As an optional embodiment, the output word segmentation vector can be multiplied with each output word table matrix respectively, and logits and sampling operations can be performed on each multiplication result to obtain a multidimensional word segmentation identifier of the output word, and then the output word segmentation can be obtained based on the multidimensional word segmentation identifier of the output word.

[0073] Figure 4 It is a decoding schematic diagram of the output word segmentation provided by the present invention, such as Figure 4 As shown, the output word segmentation vector is multiplied by the output word table matrix 1 to obtain the multiplication result 1, and after the logits and sampling operations are performed on the multiplication result 1, the word segmentation representation token id3 of the output word segmentation is obtained = 64; the output word segmentation vector is multiplied by the output word table matrix 2 to obtain the multiplication result 2, and after the logits and sampling operations are performed on the multiplication result 1, the word segmentation representation token id4 = 215 of the output word segmentation is obtained, that is, the multidimensional word segmentation identifier of the output word segmentation is (64, 215).

[0074] It should be noted that when training the above model, the input word list, the output word list and the corresponding weight matrix can be adjusted to ensure that the input word segmentation vector obtained by the current input word segmentation is consistent with the input word segmentation vector obtained by using a word list in the original model. In other words, the embodiment of the present invention does not need to retrain the original model body, but only needs to train the input word list, the output word list and the corresponding weight matrix, thereby reducing the training overhead.

[0075] For example, the input word segmentation vector corresponding to the current input word segmentation in the original model is [0.1, 0.02, 0.34, 0.82]. After using multiple word segmentation identifiers in the embodiment of the present invention to represent the current input word segmentation and encoding, the obtained input word segmentation vector should also be [0.1, 0.02, 0.34, 0.82].

[0076] The model encoding device provided by the present invention is described below. The model encoding device described below and the model encoding method described above can be referenced to each other.

[0077] Based on any of the above embodiments, Figure 5 is a schematic diagram of the structure of the model encoding device provided by the present invention, such as Figure 5 As shown, the device comprises:

[0078] A first determining unit 510, configured to determine a current input word segmentation;

[0079] The model encoding unit 520 is used to input the current input word segmentation into the model. The model performs a lookup in the corresponding input word table based on the multi-dimensional word segmentation identifier corresponding to the current input word segmentation, obtains the lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains the input word segmentation vector based on each lookup input word segmentation vector.

[0080] Based on any of the above embodiments, obtaining an input word segmentation vector based on each table lookup input word segmentation vector includes:

[0081] Combine the word segmentation vectors of each lookup table input to obtain a combined word segmentation vector;

[0082] Based on the combined word segmentation vector and the corresponding weight matrix, the input word segmentation vector is determined.

[0083] The model reasoning device provided by the present invention is described below. The model reasoning device described below and the model reasoning method described above can be referenced to each other.

[0084] Based on any of the above embodiments, Figure 6 is a schematic diagram of the structure of the model reasoning device provided by the present invention, such as Figure 6 As shown, the device comprises:

[0085] A second determining unit 610 is used to determine an input word segmentation vector based on the model encoding method described in any of the above embodiments;

[0086] The model inference unit 620 is used to perform model inference based on the input word segmentation vector to obtain an inference result.

[0087] Based on any of the above embodiments, model inference is performed based on the input word segmentation vector to obtain inference results, including:

[0088] Perform model inference based on the input word segmentation vector to obtain the output word segmentation vector;

[0089] The inference result is determined based on the output vocabulary matrix and the output word segmentation vector.

[0090] Based on any of the above embodiments, based on the output vocabulary matrix and the output word segmentation vector, determining the inference result includes:

[0091] The output word segmentation vector is multiplied by the output word list matrix, and the inference result is determined based on the multiplication result.

[0092] Figure 7 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 7As shown, the electronic device may include: a processor 710, a memory 720, a communication interface 730 and a communication bus 740, wherein the processor 710, the memory 720 and the communication interface 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 720 to execute the model encoding method, which includes: determining the current input word segmentation; inputting the current input word segmentation into the model, and the model performs a table lookup in the corresponding input word table based on the multi-dimensional word segmentation identifier corresponding to the current input word segmentation, obtaining the table lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtaining the input word segmentation vector based on each table lookup input word segmentation vector.

[0093] Or, execute a model reasoning method, which includes: determining an input word segmentation vector based on the model encoding method as described above; performing model reasoning based on the input word segmentation vector to obtain a reasoning result.

[0094] In addition, the logic instructions in the above-mentioned memory 720 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0095] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the model encoding method provided by the above-mentioned methods, and the method includes: determining the current input word segmentation; inputting the current input word segmentation into the model, and the model performs a lookup in the corresponding input word table based on the multi-dimensional word segmentation identifier corresponding to the current input word segmentation, and obtains the lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains the input word segmentation vector based on each lookup input word segmentation vector.

[0096] Or, execute a model reasoning method, which includes: determining an input word segmentation vector based on the model encoding method as described above; performing model reasoning based on the input word segmentation vector to obtain a reasoning result.

[0097] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned model encoding methods, the methods comprising: determining a current input word segmentation; inputting the current input word segmentation into a model, wherein the model performs a lookup in a corresponding input word table based on a multi-dimensional word segmentation identifier corresponding to the current input word segmentation, obtains a lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains an input word segmentation vector based on each lookup input word segmentation vector.

[0098] Or, execute a model reasoning method, which includes: determining an input word segmentation vector based on the model encoding method as described above; performing model reasoning based on the input word segmentation vector to obtain a reasoning result.

[0099] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model encoding method, characterized in that: include: Determine the current input word segmentation; The current input token is the input token of the current reasoning stage; The current input word segmentation is input into the model, and the model performs a table lookup in the corresponding input word table based on the multidimensional word segmentation identifier corresponding to the current input word segmentation, obtains the table lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains the input word segmentation vector based on each table lookup input word segmentation vector; the multidimensional word segmentation identifier corresponding to the current input word segmentation refers to the identifier of the current input word segmentation in multiple input word tables, the input word table refers to the word table corresponding to each word segmentation, and the dimension of the word segmentation identifier corresponds to the number of input word tables; The step of obtaining the input word segmentation vector based on each table lookup input word segmentation vector includes: Combine the word segmentation vectors of each lookup table input to obtain a combined word segmentation vector; Based on the combined word segmentation vector and the corresponding weight matrix, the input word segmentation vector is determined.

2. A model reasoning method, characterized in that: include: Determine an input word segmentation vector based on the model encoding method described in claim 1; Perform model reasoning based on the input word segmentation vector to obtain a reasoning result; The performing model reasoning based on the input word segmentation vector to obtain a reasoning result includes: Perform model inference based on the input word segmentation vector to obtain an output word segmentation vector; The inference result is determined based on the output vocabulary matrix and the output word segmentation vector.

3. The model reasoning method according to claim 2, characterized in that: The determining the inference result based on the output vocabulary matrix and the output word segmentation vector includes: The output word segmentation vector is multiplied by the output word list matrix, and the inference result is determined based on the multiplication result.

4. A model encoding device, characterized in that: include: A first determination unit, used to determine the current input word segmentation; The current input token is the input token of the current reasoning stage; A model encoding unit, used for inputting the current input word segmentation into the model, and the model performs a table lookup in the corresponding input word table based on the multidimensional word segmentation identifier corresponding to the current input word segmentation, obtains the table lookup input word segmentation vector corresponding to each dimensional word segmentation identifier, and obtains the input word segmentation vector based on each table lookup input word segmentation vector; the multidimensional word segmentation identifier corresponding to the current input word segmentation refers to the identifier of the current input word segmentation in multiple input word tables, the input word table refers to the word table corresponding to each word segmentation, and the dimension of the word segmentation identifier corresponds to the number of input word tables; The step of obtaining the input word segmentation vector based on each table lookup input word segmentation vector includes: Combine the word segmentation vectors of each lookup table input to obtain a combined word segmentation vector; Based on the combined word segmentation vector and the corresponding weight matrix, the input word segmentation vector is determined.

5. A model reasoning device, characterized in that: include: A second determining unit, configured to determine an input word segmentation vector based on the model encoding method according to claim 1; A model reasoning unit, used to perform model reasoning based on the input word segmentation vector to obtain a reasoning result; The performing model reasoning based on the input word segmentation vector to obtain a reasoning result includes: Perform model inference based on the input word segmentation vector to obtain an output word segmentation vector; The inference result is determined based on the output vocabulary matrix and the output word segmentation vector.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the model encoding method as described in claim 1, or implements the model reasoning method as described in any one of claims 2 to 3.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the model encoding method as claimed in claim 1, or implements the model reasoning method as claimed in any one of claims 2 to 3.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the model encoding method as claimed in claim 1, or implements the model reasoning method as claimed in any one of claims 2 to 3.

Images