A translation model compression method, translation method and related device

Abstract

One or more embodiments of the present specification disclose a translation model compression method, a translation method and a related device, the method comprising: extracting a vocabulary vector set from a target translation model, determining a quantization interval based on a normal distribution result of the vocabulary vector set mapping, calculating the number of interval segments to be divided according to the determined quantization accuracy, dividing the quantization interval into a plurality of quantization interval segments equal to the number of interval segments in an equal manner, assigning a unique interval number to each quantization interval segment, replacing the original value of the high bit corresponding to the element of the vocabulary vector contained in each quantization interval segment with the interval number of the high bit of the corresponding quantization interval segment, and writing the vocabulary vector represented by the interval number into the target translation model. Thus, compression of the target translation model is achieved.

Description

Technical Field

[0001] This document relates to the field of artificial intelligence technology, and in particular to a translation model compression method, a translation method and related devices. Background Art

[0002] With the continuous development of machine translation technology, online translation has been widely used. At the same time, with the upgrading of business, the application scenarios of offline translation are also increasing. Terminal devices such as scanning pens, translation machines, and simultaneous interpretation headphones usually use translation functions in environments with no network or weak network.

[0003] However, the memory resources of such terminal devices are generally limited, and the memory on the terminal devices is usually shared by the system module, translation module, optical character recognition module, human-computer dialogue module, etc., and the memory usage is relatively large.

[0004] At present, the industry has proposed many compression schemes for translation models. However, due to the unreasonable compression methods, although they can reduce memory usage, they can also easily cause significant losses in translation quality. Summary of the invention

[0005] The purpose of one or more embodiments of the present specification is to provide a translation model compression method, a translation method and related devices, so as to achieve compression of the translation model by means of vocabulary quantization, thereby reducing the memory occupied by the translation model while ensuring that the translation quality is minimally impaired.

[0006] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0007] First, a translation model compression method is proposed, comprising:

[0008] Extracting a vocabulary vector set from the trained target translation model, wherein elements of the vocabulary vectors in the vocabulary vector set are normally distributed;

[0009] Determine the quantization interval based on the normal distribution result of the vocabulary vector set mapping;

[0010] According to the determined quantization accuracy, the number of interval segments to be divided is calculated, the quantization interval is divided into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and a unique interval number is assigned to each quantization interval segment;

[0011] Respectively replace the original values ​​corresponding to the elements of the vocabulary vector contained in each quantization interval with the interval number of the corresponding quantization interval, and write the vocabulary vector represented by the interval number into the target translation model;

[0012] The bit position of the interval number is smaller than the bit position of the original value.

[0013] Secondly, a translation method is proposed, comprising:

[0014] Determine the target original text to be translated;

[0015] Based on the target original text, searching the target translation model for a vocabulary vector corresponding to each vocabulary in at least one vocabulary contained in the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector represented by an interval number after quantization processing using the translation model compression method described in the first aspect;

[0016] De-quantizing and determining the final value of each element in the vocabulary vector according to the correspondence between the vocabulary vector and the locally stored vector, wherein the correspondence is pre-stored by the translation model compression method described in the first aspect, and the final value of each element in the vocabulary vector determined by the de-quantization is the same as the bit position of the original value;

[0017] The final value determined by dequantization of each vocabulary included in the target original text is input into the target translation model for inference prediction, and the translation result is output.

[0018] In the third aspect, a translation model compression device is proposed, comprising:

[0019] An extraction module extracts a vocabulary vector set from the trained target translation model, wherein the elements of the vocabulary vectors in the vocabulary vector set are normally distributed;

[0020] A determination module, which determines a quantization interval based on a normal distribution result of the vocabulary vector set mapping;

[0021] A division module, which calculates the number of interval segments to be divided according to the determined quantization accuracy, divides the quantization interval into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and assigns a unique interval number to each quantization interval segment;

[0022] A quantization module replaces the original values ​​corresponding to the elements of the vocabulary vector contained in each quantization interval with the interval number of the corresponding quantization interval, and writes the vocabulary vector represented by the interval number into the target translation model;

[0023] The bit position of the interval number is smaller than the bit position of the original value.

[0024] In a fourth aspect, a translation device is proposed, comprising:

[0025] The first determination module determines the target original text to be translated;

[0026] A search module, based on the target original text, searches the target translation model for a vocabulary vector corresponding to each vocabulary in at least one vocabulary contained in the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector represented by an interval number after quantization processing using the translation model compression method described in the first aspect;

[0027] A second determination module, according to the correspondence relationship between the vocabulary vector and the locally stored vector, inverse quantization determines the final value corresponding to each element in the vocabulary vector, wherein the vector correspondence relationship is pre-stored by the translation model compression method described in the first aspect, and the final value of each element in the vocabulary vector determined by inverse quantization has the same bit as the original value;

[0028] The translation module inputs the final value determined by dequantization of each vocabulary included in the target original text into the target translation model for inference prediction, and outputs the translation result.

[0029] In a fifth aspect, an electronic device is provided, comprising:

[0030] Processor; and

[0031] A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the translation model compression method according to the first aspect, or the translation method according to the second aspect.

[0032] In the sixth aspect, a computer-readable storage medium is proposed, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the translation model compression method described in the first aspect, or the translation method described in the second aspect.

[0033] It can be seen from the technical solutions provided by one or more embodiments of the present specification that the high-bit word table vector extracted from the target translation model is quantized to obtain a low-bit word table vector to achieve the purpose of compressing the target translation model, thereby reducing the space occupied by the target translation model in the system memory. Moreover, the compression scheme does not involve changes in the model network calculation weights, and therefore has little impact on the subsequent translation quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the description of one or more embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 It is a flowchart of the steps of a translation model compression method provided in an embodiment of this specification.

[0036] Figure 2 It is a flowchart of the steps of a translation method provided in an embodiment of this specification.

[0037] Figure 3 It is a schematic diagram of the normal distribution obtained by mapping the elements in the vocabulary vector provided in the embodiments of this specification.

[0038] Figure 4 It is a structural diagram of a translation model compression device provided in an embodiment of this specification.

[0039] Figure 5 It is a structural diagram of a translation device provided in an embodiment of this specification.

[0040] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of this specification.

[0041] Figure 7 It is a structural diagram of an electronic device provided by another embodiment of the present specification. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the one or more embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0043] Based on the demand for terminal devices in offline translation business scenarios and business upgrades, the memory resource usage of terminal devices has attracted much attention. In order to reasonably plan the memory usage of terminal devices, especially for terminal devices with limited memory, it is generally inclined to compress the translation model used on the terminal device to reduce the usage of system memory resources. Although the currently proposed translation model compression scheme can reduce the usage of system memory resources to a certain extent, since the compression scheme involves network calculations within the translation model, especially network calculation weights, it will have a significant impact on the translation quality and cannot guarantee high translation quality and accuracy.

[0044] To this end, the embodiments of this specification propose a translation model compression scheme. Considering that the word table vector occupies half of the total space size of the translation model, the high-bit word table vector is quantized to obtain a low-bit word table vector to achieve the purpose of compressing the translation model, thereby reducing the occupation of the translation model in the system memory. Since the word table vector has little impact on the network calculation inside the translation model and almost does not involve changes in the network calculation weights, it has little impact on the final translation quality.

[0045] It should be understood that in the embodiments of this specification, quantizing the vocabulary vector refers to converting the high-bit vocabulary vector into a low-bit vocabulary vector, thereby achieving lightweighting of the vocabulary vector and further lightweighting the translation model.

[0046] The technical solutions involved in this specification are described in detail below through specific embodiments.

[0047] Embodiment 1

[0048] Reference Figure 1 As shown, it is a flowchart of the steps of a translation model compression method provided by an embodiment of this specification. The execution subject of the compression method can be a hardware device (for example, a smart phone, a personal computer, a wearable device, a tablet computer, etc.) or a software device (for example, a client or a software module combination integrated in the aforementioned hardware devices) with certain computing and processing capabilities, and this specification does not limit this. The translation model compression method may include the following steps:

[0049] Step 102: extracting a vocabulary vector set from the trained target translation model, wherein elements of the vocabulary vectors in the vocabulary vector set are normally distributed.

[0050] Words Embedding is the matrix value of the source word list and the target word list in the translation model after vectorization. In a trained target translation model, the number of words contained may be large, and accordingly, the number of word vectors is also large. Therefore, these word vectors can be combined into a word vector set and stored in the target translation model.

[0051] It should be understood that a vocabulary vector can contain M feature dimensions (i.e., elements). If there are 6,000 vocabularies in the target translation model, and the vocabulary vector corresponding to each vocabulary is 512-dimensional, then the target model will have a feature dimension consisting of 6,000×512 elements. Mapping these vocabulary vectors to the coordinate system, the distribution intervals of the elements corresponding to the vocabulary vectors are relatively concentrated, and generally present a normal distribution. In fact, the normal distribution referred to here is not a standard normal distribution, but a trend. The distribution of the elements contained in these vocabulary vectors tends to be normally distributed. In the embodiments of this specification, the normal distribution is used as an example for explanation.

[0052] Step 104: Determine a quantization interval based on a normal distribution result of the vocabulary vector set mapping.

[0053] In an embodiment of the present specification, a vocabulary vector distribution interval that satisfies preset constraints can be selected as a quantization interval based on the normal distribution of the vocabulary vector set mapping. The preset constraints can be flexibly set according to business needs. The embodiments of the present specification take the selection of all vocabulary vectors in the vocabulary vector set and the selection of the degree of concentration of vocabulary vectors that satisfies the constraint sub-condition as different examples of preset constraints, and select the vocabulary vector distribution interval that satisfies the preset constraints from the normal distribution results as the quantization interval.

[0054] --The preset constraint 1 is to select the distribution interval corresponding to all the vocabulary vectors in the vocabulary vector set.

[0055] Correspondingly, when a vocabulary vector distribution interval that satisfies preset constraint 1 is selected as a quantization interval based on the normal distribution of the vocabulary vector set mapping, the vocabulary vector distribution interval corresponding to all vocabulary vectors can be selected as the quantization interval based on the normal distribution of the vocabulary vector set mapping, wherein the maximum boundary of the quantization interval is the original value of the largest element in the vocabulary vector set, and the minimum boundary is the original value of the smallest element in the vocabulary vector set.

[0056] For example, assuming that in the normal distribution result obtained by mapping the vocabulary vector set, the vocabulary vector distribution interval of the elements in the vocabulary vector set is [-0.809, 0.758], then if the preset constraint condition 1 is used as the determination condition, the vocabulary vector distribution interval corresponding to all the vocabulary vectors in the vocabulary vector set is selected, that is, [-0.809, 0.758] is used as the quantization interval. Thus, the vocabulary vectors in the entire vocabulary vector set are subjected to subsequent quantization compression processing.

[0057] Although the quantization interval determined by the preset constraint condition 1 can cover all the vocabulary vectors in the vocabulary vector set, expand the compression range, improve the compression effect to a greater extent, and reduce the translation model occupancy problem. However, considering that there are many vocabulary vectors contained in the vocabulary vector set, and there are many elements in each vocabulary vector, and the elements in these vocabulary vectors are normally distributed, most of the elements are concentrated in a certain distribution interval. Therefore, it is possible to consider using the preset constraint condition 2 as the selection condition, and select the distribution interval with a relatively concentrated degree of vocabulary vector concentration from the normal distribution result as the quantization interval, so as to reasonably determine the quantization interval that needs to be compressed, avoid inefficient compression caused by too large a range of compressed quantization intervals, and reduce the precision loss caused by the dispersed distribution of elements in the compressed quantization interval.

[0058] --The preset constraint condition 2 is to select the vocabulary vector distribution interval whose degree of vocabulary vector concentration satisfies the constraint sub-condition.

[0059] Accordingly, when a vocabulary vector distribution interval that satisfies preset constraints is selected as a quantization interval based on the normal distribution of the vocabulary vector set mapping, a vocabulary vector distribution interval whose degree of vocabulary vector concentration satisfies constraint sub-conditions can be selected as a quantization interval based on the normal distribution of the vocabulary vector set mapping; wherein the constraint sub-conditions may include: the value of the element at the maximum boundary of the vocabulary vector distribution interval satisfies a first threshold and / or the value of the element at the minimum boundary satisfies a second threshold; or, the density of the elements at the maximum boundary of the vocabulary vector distribution interval satisfies a third threshold and / or the density of the elements at the minimum boundary satisfies a fourth threshold.

[0060] For example, assuming that in the normal distribution result obtained by mapping the vocabulary vector set, the vocabulary vector distribution interval of the elements in the vocabulary vector set is [-0.809, 0.758]. Then, if the preset constraint condition 2 is used as the determination condition, when the constraint sub-condition is that the value of the element at the maximum boundary of the quantization interval satisfies the first threshold, the determined quantization interval can be [-0.5, 0.758], or, when the constraint sub-condition is that the value of the element at the minimum boundary of the quantization interval satisfies the second threshold, the determined quantization interval can be [-0.809, 0.6], or, when the constraint sub-condition is that the value of the element at the maximum boundary of the quantization interval satisfies the first threshold and the value of the element at the minimum boundary satisfies the second threshold, the determined quantization interval is [-0.5, 0.6]. The first threshold here takes -0.5 as an example, and the second threshold takes 0.6 as an example. In fact, the first threshold and the second threshold here can be determined based on the concentration of element distribution determined by analyzing the normal distribution results. Generally, most elements in the vocabulary vector set are distributed in [-0.5, 0.5] by default. The elements in this interval are more concentrated, accounting for about 60%-80% of all vocabulary vectors, while the elements in other distribution intervals are more dispersed. Therefore, the first threshold and the second threshold can be determined by this analysis.

[0061] When the constraint sub-condition is that the density of elements at the maximum boundary of the vocabulary vector distribution interval meets the third threshold, the quantization interval can be determined to be [-0.6, 0.758], or, when the constraint sub-condition is that the density of elements at the minimum boundary of the vocabulary vector distribution interval meets the fourth threshold, the quantization interval can be determined to be [-0.809, 0.7], or, when the constraint sub-condition is that the density of elements at the maximum boundary of the vocabulary vector distribution interval meets the third threshold, and the density of elements at the minimum boundary meets the fourth threshold, the quantization interval can be determined to be [-0.6, 0.7]. The third threshold here is taken as an example of -0.6, and the fourth threshold is taken as an example of 0.7. In fact, the third threshold and the fourth threshold here can be determined according to the concentration of element distribution determined by the analysis of the normal distribution results. In general, most elements in the vocabulary vector set are concentrated in [-0.6, 0.7]. In this interval, the density of elements at each node is relatively large, for example, occupying more than 20% of the total elements, while in the distribution interval outside this interval, the density of elements at each node is small and scattered.

[0062] It should be understood that in the embodiments of this specification, the values ​​of the first threshold and the second threshold may be equal or different. Specifically, it can be determined according to the distribution of elements of the vocabulary vector set in each translation model to be quantized, and no specific limitation is made here. Similarly, the third threshold and the fourth threshold may also be equal or different. That is, the quantization interval is a symmetrical distribution interval, or an asymmetrical distribution interval.

[0063] It should be emphasized here that the first threshold, the second threshold, the third threshold and the fourth threshold can be set according to empirical values, and can also be flexibly adjusted according to the current business needs, as long as the element distribution of the quantization interval determined by the threshold (which can be simply and violently understood as the vocabulary vector distribution) is relatively concentrated.

[0064] Step 106: Calculate the number of interval segments to be divided according to the determined quantization accuracy, divide the quantization interval into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and assign a unique interval number to each quantization interval segment.

[0065] Usually, the precision of the word table vector after the translation model is trained is a 32-bit floating point number, and the quantization process is to map the 32-bit high precision into a low-precision value, such as an 8-bit or 16-bit integer. The precision loss and memory compression effect caused by different quantization precisions are also different. The lower the quantization precision, the more memory is saved, but the quality loss is usually greater. In the embodiment of this specification, the quantization precision can be based on the precision of the original value of each element of the word table vector in the currently trained translation model, and the precision smaller than the bit of the current original value can be reasonably selected as the quantization precision. The specific selection method is not limited and can be flexibly selected according to needs.

[0066] In the embodiments of this specification, after the quantization interval is determined, the quantization interval can be equally divided into a plurality of quantization interval segments based on the number of interval segments calculated based on the quantization accuracy. At the same time, an interval number is assigned to each quantization interval segment to uniquely identify the quantization interval segment. The interval number can be a number or a character or a letter or a different combination of numbers, characters, and letters. In an optional implementation, the interval number is a number, and consecutive interval segments are respectively identified by consecutive numbers.

[0067] The specific implementation of step 106 is described in detail below based on the quantization interval determined by the above preset constraint conditions as an example.

[0068] Corresponding to the quantization interval determined based on the preset constraint condition 1, the number of interval segments is defined as N, and N is a positive integer; the determined quantization accuracy can be calculated as the exponent of the base 2, and the number of interval segments to be divided N is obtained by subtracting 1 from the calculation result; the quantization interval is evenly divided into N quantization interval segments in an equal manner, and each positive integer from 0 to N-1 is used to assign an interval number to each quantization interval segment.

[0069] For example, assuming that the original value of the element is a 32-bit floating point number, the quantization precision can be determined as an 8-bit integer, and the number of calculation intervals is 2. 8-1=255. Then the quantization interval can be evenly divided into 255 quantization interval segments, and each quantization interval segment is assigned an interval number using 0-254. For example, starting from left to right, the first quantization interval segment is numbered 0, the second quantization interval segment is numbered 1, the third quantization interval segment is numbered 2..., and the 255th quantization interval segment is numbered 254.

[0070] Corresponding to the quantization interval determined based on the preset constraint condition 2, the number of interval segments is also defined as N, and N is a positive integer; the determined quantization accuracy can be calculated as the exponent of the base 2, and the number of interval segments to be divided N is obtained by subtracting 1 from the calculation result; the quantization interval is evenly divided into N quantization interval segments in an equal manner, and each positive integer from 0 to N-1 is used to assign an interval number to each quantization interval segment; and the distribution intervals outside the quantization interval are uniformly assigned a special number N.

[0071] For example, assuming that the original value of the element is a 32-bit floating point number, the quantization precision can be determined as an 8-bit integer, and the number of calculation intervals is 2. 8 -1=255. Then the quantization interval can be evenly divided into 255 quantization interval segments, and each quantization interval segment can be assigned an interval number from 0 to 254. Afterwards, considering that the quantization interval determined by the preset constraint condition 2 is only a partial distribution interval, it is also necessary to identify the distribution interval that does not need to be quantized. In this way, 255 can be used as a special number to uniformly identify the distribution intervals outside the quantization interval. For example, starting from left to right in the quantization interval, the interval number of the first quantization interval segment is 0, the interval number of the second quantization interval segment is 1, the interval number of the third quantization interval segment is 2..., and the interval number of the 255th quantization interval segment is 254. The distribution intervals outside the quantization interval are uniformly identified as the special number 255.

[0072] Step 108: Replace the original values ​​corresponding to the elements of the vocabulary vector contained in each quantization interval with the interval number of the corresponding quantization interval, and write the vocabulary vector represented by the interval number into the target translation model; wherein the bit position of the interval number is smaller than the bit position of the original value.

[0073] After assigning interval numbers to each equally divided quantized interval segment, the original values ​​corresponding to the elements of the vocabulary vector contained in each quantized interval segment can be replaced with the interval number of the corresponding quantized interval segment. For example, if an element of the vocabulary vector (original value: 0.3456) is in the quantized interval segment with interval number 128, the element of the vocabulary vector is replaced with 128 for representation, and all vocabulary vectors represented by interval numbers are written into the target translation model. At this time, the vocabulary vector in the target translation model has been replaced by an integer from the original floating point number, and the bit position is reduced. Therefore, the vocabulary vector is subjected to a certain compression processing, and then the target translation model is compressed, which also reduces the space occupied by the target translation model in the system memory. Moreover, as mentioned above, the scheme of the compressed translation model does not involve network calculation weights, and therefore will not have too much impact on the computing network, thereby ensuring that the translation quality loss after compression is within the tolerable range.

[0074] Further, in the embodiments of the present specification, for the case where the quantization interval is determined using the preset constraint 2, after the vocabulary vector represented by the interval number is written into the target translation model, for the distribution interval outside the quantization interval, the vocabulary vectors contained in the distribution interval outside the quantization interval can be written into the target translation model with the corresponding original values, and uniformly marked using a special number N. That is, if the quantization interval is [-0.5, 0.5], then the elements in the vocabulary vectors of the distribution intervals (-∞, -0.5) and (0.5, +∞) are rewritten into the target translation model according to the original values, that is, no quantization is performed. However, the vocabulary vectors falling into these distribution intervals can be uniformly marked with a special number, for example, the 255 mentioned above. In fact, other numbers can also be used, as long as the number is defined in the method as a distribution interval outside the specially marked quantization interval.

[0075] It should be understood that in the embodiments of the present specification, the above-mentioned quantization compression scheme is to convert the high-bit word table vector into a low-bit word table vector. Then, when the word table vector is subsequently used for network calculation, the word table vector represented by the low-bit interval number in the translation model can be dequantized back to the original bit word table vector. Therefore, after the word table vector represented by the interval number is written into the target translation model, the correspondence between each quantization interval segment and the assigned interval number can be saved, wherein the quantization interval segment is a word table vector distribution interval with the original value corresponding to the element of the word table vector as the boundary.

[0076] It should be noted that the above correspondence relationship can be stored in a storage space other than the system memory, that is, it will not be stored in the translation model. Specifically, it can be stored in a relational table or non-table manner, which is not limited in the embodiments of this specification.

[0077] Through the above technical solution, the high-bit word table vector extracted from the target translation model is quantized to obtain the low-bit word table vector, so as to achieve the purpose of compressing the target translation model, thereby reducing the space occupied by the target translation model in the system memory. Moreover, the compression scheme does not involve the change of the model network calculation weight, so it has little impact on the subsequent translation quality.

[0078] Embodiment 2

[0079] Reference Figure 2 As shown, it is a flowchart of the steps of a translation method provided by an embodiment of this specification. The execution subject of the translation method can be a hardware device with certain computing and processing capabilities (for example, a smart phone, a personal computer, a wearable device, a tablet computer, etc.) or a software device (for example, a client or a combination of software modules integrated on the aforementioned types of hardware devices). In fact, the execution subject of the translation method can be the same as the execution subject of the translation model compression method in the first embodiment, or it can be different, and this specification does not limit this. The translation method may include the following steps:

[0080] Step 202: Determine the target original text to be translated.

[0081] The target original text may be any segment of the same language type as the source text contained in the vocabulary vector in the target translation model.

[0082] Step 204: Based on the target original text, search the target translation model for a vocabulary vector corresponding to each vocabulary in at least one vocabulary contained in the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector represented by an interval number after quantization processing using the translation model compression method described in steps 102 to 108.

[0083] The specific implementation of step 204 can be performed according to the existing vocabulary search method, and the vocabulary vector corresponding to each vocabulary included in the target original text is searched from the target translation model in turn. The vocabulary vector stored in the target translation model is obtained by the quantization compression method described in Example 1. The specific description is referred to in Example 1, and will not be repeated here.

[0084] Step 206: According to the correspondence between the vocabulary vector and the locally stored relationship, the final value of each element in the vocabulary vector is determined by inverse quantization, wherein the correspondence is pre-saved by the translation model compression method described in steps 102 to 108, and the final value of each element in the vocabulary vector determined by inverse quantization is the same as the bit of the original value.

[0085] After finding the word vector corresponding to the word list contained in the target original text, the final value of each element in the word vector can be determined by inverse quantization search based on the corresponding relationship stored locally in the word vector. The corresponding relationship involved is determined based on the translation model compression scheme described in Example 1. Please refer to the description in Example 1 for details, which will not be repeated here.

[0086] In an embodiment of the present specification, when the final value corresponding to each element in the word list vector is determined by inverse quantization according to the correspondence between the word list vector and the locally stored data, if the word list vector is represented by an interval number, the quantization interval segment that matches the interval number corresponding to each element in the word list vector can be found from the locally stored correspondence based on the word list vector; the node value in the quantization interval segment that meets the preset rules is determined as the final value of the element corresponding to the interval number. Optionally, the node value that meets the preset rules can be the maximum boundary value, or the minimum boundary value, or the median of the quantization interval segment. In fact, the node value that meets the preset rules can also be any other value in the quantization interval segment, which is not limited here. It should be understood that in order to facilitate the determination of the final value, the node values ​​that meet the preset rules can be unified for all quantization interval segments, for example, the median of each quantization interval segment is uniformly used as the final value of the current quantization interval segment.

[0087] It should be emphasized that the bits of the final value determined by dequantization are the same as the bits of the original value. For example, the original value of the element of the vocabulary vector is a 32-bit floating point number, which is represented by an 8-bit integer interval number after quantization, and is represented by a 32-bit floating point final value after dequantization. It can be seen that although there may be a certain error between the original value and the final value of the element of the vocabulary vector after dequantization, it does not have a significant impact on the predicted distribution of the later inference calculation of the model.

[0088] If the vocabulary vector is marked with a special number, the original value of each element in the vocabulary vector is directly determined as the final value of each element in the vocabulary vector.

[0089] Step 208: inputting the final value determined by dequantization of each vocabulary included in the target original text into the target translation model for inference prediction, and outputting the translation result.

[0090] Through the above technical solution, during the translation process, the corresponding relationship saved during the previous quantization process is used to dequantize the low-bit word table vector found back to the high-bit final value. Although the final value may have a certain error with the original value, the final value has the same bit as the original value. After being input into the translation model, a large number of final values ​​will participate in the network model calculation. The error can be ignored, and the prediction result obtained by inference prediction will not have too much deviation. Therefore, when the translation model is compressed, the translation quality loss is guaranteed to be small, especially for memory-constrained devices, which can improve the reasonable allocation of overall memory and improve translation efficiency.

[0091] Embodiment 3

[0092] The above translation model compression scheme and translation scheme are described in detail below using a specific example.

[0093] Assuming that the target translation model has been trained, the distribution characteristics of elements in the vocabulary vector in the target translation model meet the following requirements: normal distribution, clustered around 0; small distribution range, and concentrated distribution.

[0094] Reference Figure 3 As shown, first, map the elements in the vocabulary vector to Figure 3 In the coordinate system shown, the vocabulary vectors are distributed in an open interval. In actual projects, they are concentrated between [-0.5, 0.5]. Generally, a relatively concentrated interval is selected as the quantization interval, which will reduce the loss of quantization accuracy caused by data discreteness.

[0095] Usually, the accuracy of the vocabulary vector after model training is 32-bit floating point number, and the quantization process is to map the 32-bit high precision into a low precision value, such as 8-bit or 16-bit integer. The precision loss and memory compression effect caused by different quantization precision are also different. The lower the quantization precision, the more memory is saved, but the quality loss is usually greater.

[0096] Quantification process:

[0097] Here we can select [-0.5, 0.5] as the quantization interval. Here we can determine the quantization precision to be an 8-bit integer. Divide the quantization interval [-0.5, 0.5] into 2 8 -1 = 255, resulting in 255 quantization intervals. Each quantization interval is assigned an interval number index, for example, 1-254. Other distribution intervals other than the quantization interval [-0.5, 0.5] are assigned a special number 255.

[0098] For the elements of the vocabulary vector in the quantization interval, the original value w is replaced with the interval number index and saved to the translation model. For example, int8(index) replaces fp32(w) and saves it to the translation model. As shown in the figure, int8(102) replaces fp32(-0.1) and writes it to the translation model; for another example, int(204) replaces fp32(0.3) and writes it to the translation model.

[0099] For the elements of the vocabulary vector outside the quantization interval, the original value w is directly stored in the translation model and marked with a special number index. For example, fp32(w) is directly saved to the translation model and marked with index, where the marked index is 255.

[0100] In this way, the quantization process of the vocabulary vector is completed, that is, the compression of the translation model is achieved.

[0101] Dequantization process:

[0102] After determining the target original text to be translated, the word table vector of the word table in the target original text is searched from the above-mentioned compressed translation model. In actual calculation, the final value of the fp32 bit can be dequantized according to the index value; if the index is a special number 255, the original value of the fp32 word table vector can be directly read as the final value. If it is not a special number, the final value of the fp32 bit of the corresponding quantization interval segment is taken out according to the value of the index. This final value can be the left boundary, the right boundary, or the median of the interval. This final value will have a certain error with the original value, but it has little effect on the prediction analysis of the later reasoning calculation of the translation model, that is, it has little effect on the bilingual evaluation index bleu value of the final translation result, but it saves a lot of memory. Quantization from fp32 to int8 can save nearly 3 / 4 of the memory, which is very useful for devices with limited memory resources. Moreover, in actual business, if the word table vector is more concentrated, the determined quantization interval will be smaller, and then the overall translation quality loss will be smaller.

[0103] In the process of implementing the above technical solution, based on the distribution concentration characteristics of the elements in the vocabulary vector and the characteristics of the translation model, the vocabulary vector is quantized to compress the translation model, and the compressed translation model is dequantized for inference and prediction. The entire process does not make any changes to the network weights of the translation model, which can achieve significant compression of the translation model. At the same time, the loss of translation quality is almost negligible. The entire compression process is relatively easy to implement and effectively avoids the defect of traditional compression schemes that cause large loss of translation quality, which can also improve translation efficiency as a whole.

[0104] Embodiment 4

[0105] Reference Figure 4 As shown, it is a structural schematic diagram of a translation model compression device provided in an embodiment of this specification. The translation model compression device 400 may include:

[0106] Extraction module 402 extracts a vocabulary vector set from the trained target translation model, wherein the elements of the vocabulary vectors in the vocabulary vector set are normally distributed.

[0107] The determination module 404 determines a quantization interval based on a normal distribution result of the vocabulary vector set mapping.

[0108] The division module 406 calculates the number of interval segments to be divided according to the determined quantization accuracy, divides the quantization interval into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and assigns a unique interval number to each quantization interval segment.

[0109] The quantization module 408 replaces the original values ​​corresponding to the elements of the vocabulary vector contained in each quantization interval with the interval number of the corresponding quantization interval, and writes the vocabulary vector represented by the interval number into the target translation model; wherein the bit position of the interval number is smaller than the bit position of the original value.

[0110] Optionally, as an embodiment, when determining the quantization interval based on the normal distribution result of the vocabulary vector set mapping, the determination module 404 is specifically used to select the vocabulary vector distribution interval that meets the preset constraints as the quantization interval based on the normal distribution of the vocabulary vector set mapping.

[0111] In a specific implementation of the embodiments of the present specification, the preset constraints include: all vocabulary vectors in the vocabulary vector set; then the determination module 404 selects the vocabulary vector distribution interval that meets the preset constraints as the quantization interval in the normal distribution based on the mapping of the vocabulary vector set, and is specifically used to select the vocabulary vector distribution interval corresponding to all vocabulary vectors as the quantization interval based on the normal distribution of the mapping of the vocabulary vector set, wherein the maximum boundary of the quantization interval is the original value of the largest element in the vocabulary vector set, and the minimum boundary is the original value of the smallest element in the vocabulary vector set.

[0112] In another specific implementation of the embodiments of this specification, the preset constraint condition includes: the degree of concentration of the vocabulary vector satisfies the constraint sub-condition; then when the determination module 404 selects the vocabulary vector distribution interval that satisfies the preset constraint condition as the quantization interval in the normal distribution based on the mapping of the vocabulary vector set, it is specifically used to select the vocabulary vector distribution interval whose degree of concentration satisfies the constraint sub-condition as the quantization interval based on the normal distribution of the vocabulary vector set mapping; wherein the constraint sub-condition includes: the value of the element at the maximum boundary of the vocabulary vector distribution interval satisfies the first threshold and / or the value of the element at the minimum boundary satisfies the second threshold; or, the density of the elements at the maximum boundary of the vocabulary vector distribution interval satisfies the third threshold and / or the density of the elements at the minimum boundary satisfies the fourth threshold.

[0113] In another specific implementation of the embodiments of the present specification, the number of interval segments is N, and N is a positive integer; then the division module 406 calculates the number of interval segments to be divided according to the determined quantization accuracy, divides the quantization interval into a plurality of quantization interval segments equal to the number of interval segments in an equal manner, and assigns a unique interval number to each quantization interval segment, and is specifically used to calculate the determined quantization accuracy as the exponent of the base 2, and obtain the number of interval segments to be divided N after subtracting 1 from the calculation result; divides the quantization interval evenly into N quantization interval segments in an equal manner, and uses each positive integer from 0 to N-1 to assign an interval number to each quantization interval segment.

[0114] In another specific implementation of the embodiments of the present specification, the number of interval segments is N, and N is a positive integer; then the division module 406 calculates the number of interval segments to be divided according to the determined quantization accuracy, divides the quantization interval into a plurality of quantization interval segments equal to the number of interval segments in an equal manner, and assigns a unique interval number to each quantization interval segment, and is specifically used to calculate the determined quantization accuracy as the exponent of the base 2, and obtain the number of interval segments to be divided N after subtracting 1 from the calculation result; divides the quantization interval into N quantization interval segments in an equal manner, and assigns an interval number to each quantization interval segment using each positive integer from 0 to N-1; and uniformly assigns a special number N to the distribution intervals outside the quantization interval.

[0115] In another specific implementation of the embodiments of the present specification, the translation model compression device also includes: a marking module, which is used to write the vocabulary vectors contained in the distribution intervals outside the quantization interval into the target translation model with corresponding original values ​​after the quantization module 408 writes the vocabulary vector represented by the interval number into the target translation model, and uses a special number N for uniform marking.

[0116] In another specific implementation of the embodiments of this specification, the quantization interval is a symmetrical distribution interval, or an asymmetrical distribution interval.

[0117] In another specific implementation of the embodiments of the present specification, the translation model compression device also includes: a saving module, which is used to save the correspondence between each quantization interval segment and the assigned interval number after the quantization module 408 writes the vocabulary vector represented by the interval number into the target translation model, wherein the quantization interval segment is a vocabulary vector distribution interval with the original value corresponding to the element of the vocabulary vector as the boundary.

[0118] Through the above technical solution, based on the translation model compression device, the high-bit word list vector in the translation model can be quantized into a low-bit representation to achieve the purpose of compressing the translation model and reduce the space occupied by the translation model in the system memory. Especially for devices with limited memory, the memory usage optimization is more obvious, which improves the efficiency of machine translation. Moreover, the compression scheme has little or no effect on the model network calculation weights. Therefore, it can ensure that the translation quality loss is small while achieving the compression of the translation model.

[0119] Embodiment 5

[0120] Reference Figure 5 FIG. 5 is a schematic diagram of a translation device provided in an embodiment of the present specification. The translation device 500 may include:

[0121] The first determination module 502 determines the target original text to be translated;

[0122] A search module 504 searches for a vocabulary vector corresponding to each vocabulary in at least one vocabulary included in the target original text from a target translation model based on the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector represented by an interval number after quantization processing using the translation model compression method described in the first embodiment;

[0123] A second determination module 506, according to the correspondence relationship between the vocabulary vector and the locally stored vector, inverse quantization determines the final value corresponding to each element in the vocabulary vector, wherein the vector correspondence relationship is pre-stored by the translation model compression method described in the first embodiment, and the final value of each element in the vocabulary vector determined by inverse quantization has the same bit as the original value;

[0124] The translation module 508 inputs the final value determined by dequantization of each vocabulary included in the target original text into the target translation model for inference prediction, and outputs the translation result.

[0125] Optionally, as an embodiment, when the second determination module 506 inversely quantizes and determines the final value corresponding to each element in the vocabulary vector according to the correspondence between the vocabulary vector and the locally stored correspondence, it is specifically used to search for a quantization interval segment that matches the interval number corresponding to each element in the vocabulary vector based on the correspondence between the vocabulary vector and the locally stored correspondence; and determine the node value in the quantization interval segment that meets the preset rules as the final value of the element corresponding to the interval number.

[0126] In a specific implementation of the embodiments of the present specification, the node value that complies with the preset rule is the maximum boundary value, or the minimum boundary value, or the median of the quantization interval segment.

[0127] It should be understood that in the embodiments of this specification, Figure 5 The illustrated translation device 500 may be used with Figure 4 The translation model compression device 400 shown belongs to the same device, or different devices, and the two can also be integrated into the same electronic device to collaboratively perform translation services. The embodiments of this specification do not limit the relationship between the two.

[0128] Through the above technical scheme, the translation device is used to search the target original text in the vocabulary, and the vocabulary vector represented by the low bit precision after quantization is dequantized to the final value of the high bit, and the vocabulary vector represented by the final value is used to input the translation model for inference prediction, and the translation result is output. Since the vocabulary vector of the translation model used in the translation process is quantized, it does not occupy much system memory space, and provides enough system memory for inference prediction. Although the high bit final value obtained after dequantization is not the original value of the element in the vocabulary vector, the bit of the final value and the original value is the same, and the number of elements contained in each vocabulary vector is relatively large, therefore, the difference between the final value and the original value can be basically ignored when the model network calculation is performed later. In this way, the final inference prediction of the translation result will not have too much deviation, ensuring that the translation quality loss is small.

[0129] Embodiment 6

[0130] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this specification. Figure 6 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0131] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0132] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0133] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a translation model compression device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0134] A vocabulary vector set is extracted from a target translation model obtained through training, wherein the elements of the vocabulary vectors in the vocabulary vector set are normally distributed; a quantization interval is determined based on the normal distribution result mapped by the vocabulary vector set; the number of interval segments to be divided is calculated according to the determined quantization accuracy, the quantization interval is divided into a plurality of quantization interval segments which are the same in number as the interval segments in an equal manner, and a unique interval number is assigned to each quantization interval segment; the original values ​​corresponding to the elements of the vocabulary vectors contained in each quantization interval segment are replaced with the interval number of the corresponding quantization interval segment, and the vocabulary vector represented by the interval number is written into the target translation model; wherein the bit position of the interval number is smaller than the bit position of the original value.

[0135] The above is as in this manual Figure 1 and Figure 3The method performed by the device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in one or more embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in one or more embodiments of this specification can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0136] The electronic device may also perform Figure 1 and Figure 3 The method and the corresponding device are implemented in Figure 1 and Figure 3 The functions of the illustrated embodiments will not be described in detail in the embodiments of this specification.

[0137] Of course, in addition to software implementation, the electronic device of the embodiments of this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0138] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this specification. Figure 7At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0139] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0140] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0141] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a translation device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0142] Determine a target original text to be translated; search the target translation model for a vocabulary vector corresponding to each vocabulary in at least one vocabulary contained in the target original text based on the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector represented by an interval number after quantization processing using the translation model compression method described in Example 1; according to the correspondence between the vocabulary vector and the locally stored one, inverse quantization is performed to determine the final value of each element in the vocabulary vector, wherein the correspondence is pre-saved by the translation model compression method described in Example 1, and the final value of each element in the vocabulary vector determined by inverse quantization is the same as the bit of the original value; input the final value of each vocabulary contained in the target original text determined by inverse quantization into the target translation model for inference prediction, and output the translation result.

[0143] The above is as in this manual Figure 2 and Figure 3The method performed by the device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in one or more embodiments of this specification can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in one or more embodiments of this specification can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0144] The electronic device may also perform Figure 2 and Figure 3 The method and the corresponding device are implemented in Figure 2 and Figure 3 The functions of the illustrated embodiments will not be described in detail in the embodiments of this specification.

[0145] Of course, in addition to software implementation, the electronic device of the embodiments of this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0146] The embodiment of the present specification also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by a portable electronic device including a plurality of application programs, enable the portable electronic device to execute Figure 1-3 The method of the embodiment shown is specifically used to perform the following method:

[0147] A vocabulary vector set is extracted from a target translation model obtained through training, wherein the elements of the vocabulary vectors in the vocabulary vector set are normally distributed; a quantization interval is determined based on the normal distribution result mapped by the vocabulary vector set; the number of interval segments to be divided is calculated according to the determined quantization accuracy, the quantization interval is divided into a plurality of quantization interval segments which are the same in number as the interval segments in an equal manner, and a unique interval number is assigned to each quantization interval segment; the original values ​​corresponding to the elements of the vocabulary vectors contained in each quantization interval segment are replaced with the interval number of the corresponding quantization interval segment, and the vocabulary vector represented by the interval number is written into the target translation model; wherein the bit position of the interval number is smaller than the bit position of the original value.

[0148] or;

[0149] Determine a target original text to be translated; search the target translation model for a vocabulary vector corresponding to each vocabulary in at least one vocabulary contained in the target original text based on the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector represented by an interval number after quantization processing using the translation model compression method described in Example 1; according to the correspondence between the vocabulary vector and the locally stored one, inverse quantization is performed to determine the final value of each element in the vocabulary vector, wherein the correspondence is pre-saved by the translation model compression method described in Example 1, and the final value of each element in the vocabulary vector determined by inverse quantization is the same as the bit of the original value; input the final value of each vocabulary contained in the target original text determined by inverse quantization into the target translation model for inference prediction, and output the translation result.

[0150] In short, the above description is only a preferred embodiment of this specification and is not intended to limit the protection scope of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the protection scope of this specification.

[0151] The systems, devices, modules or units described in one or more of the above embodiments may be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0152] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0153] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0154] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0155] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A translation model compression method, comprising: Extracting a vocabulary vector set from the trained target translation model, wherein elements of the vocabulary vectors in the vocabulary vector set are normally distributed; Determine the quantization interval based on the normal distribution result of the vocabulary vector set mapping; According to the determined quantization accuracy, the number of interval segments to be divided is calculated, the quantization interval is divided into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and a unique interval number is assigned to each quantization interval segment; Respectively replace the original values ​​corresponding to the elements of the vocabulary vector contained in each quantization interval with the interval number of the corresponding quantization interval, and write the vocabulary vector represented by the interval number into the target translation model; The bit position of the interval number is smaller than the bit position of the original value.

2. The translation model compression method according to claim 1, wherein the quantization interval is determined based on the normal distribution result of the vocabulary vector set mapping, comprising: Based on the normal distribution of the vocabulary vector set mapping, a vocabulary vector distribution interval that meets preset constraints is selected as a quantization interval.

3. The translation model compression method according to claim 2, wherein the preset constraint condition comprises: All vocabulary vectors in the vocabulary vector set; Based on the normal distribution of the vocabulary vector set mapping, a vocabulary vector distribution interval that meets the preset constraint condition is selected as the quantization interval, including: Based on the normal distribution of the vocabulary vector set mapping, the vocabulary vector distribution interval corresponding to all vocabulary vectors is selected as the quantization interval, wherein the maximum boundary of the quantization interval is the original value of the largest element in the vocabulary vector set, and the minimum boundary is the original value of the smallest element in the vocabulary vector set.

4. The translation model compression method according to claim 2, wherein the preset constraint condition comprises: The concentration degree of the vocabulary vector satisfies the constraint sub-condition; Based on the normal distribution of the vocabulary vector set mapping, a vocabulary vector distribution interval that meets the preset constraint condition is selected as the quantization interval, including: Based on the normal distribution of the vocabulary vector set mapping, select a vocabulary vector distribution interval whose degree of vocabulary vector concentration satisfies the constraint sub-condition as a quantization interval; Among them, the constraint sub-conditions include: the value of the element at the maximum boundary of the vocabulary vector distribution interval satisfies the first threshold and / or the value of the element at the minimum boundary satisfies the second threshold; or, the density of the elements at the maximum boundary of the vocabulary vector distribution interval satisfies the third threshold and / or the density of the elements at the minimum boundary satisfies the fourth threshold.

5. The translation model compression method according to any one of claims 1 to 3, wherein the number of interval segments is N, and N is a positive integer; Then, according to the determined quantization accuracy, the number of interval segments to be divided is calculated, the quantization interval is divided into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and a unique interval number is assigned to each quantization interval segment, including: The determined quantization accuracy is used as the exponent of the base 2 for calculation, and the number of interval segments to be divided N is obtained by subtracting 1 from the calculation result; The quantization interval is evenly divided into N quantization interval segments in an equal manner, and each positive integer from 0 to N-1 is used to assign an interval number to each quantization interval segment.

6. The translation model compression method according to any one of claims 1 to 2 and 4, wherein the number of interval segments is N, and N is a positive integer; Then, according to the determined quantization accuracy, the number of interval segments to be divided is calculated, the quantization interval is divided into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and a unique interval number is assigned to each quantization interval segment, including: The determined quantization accuracy is used as the exponent of the base 2 for calculation, and the number of interval segments to be divided N is obtained by subtracting 1 from the calculation result; The quantization interval is evenly divided into N quantization interval segments in an equal manner, and each positive integer from 0 to N-1 is used to assign an interval number to each quantization interval segment; and a special number N is uniformly assigned to the distribution intervals outside the quantization interval.

7. The translation model compression method according to claim 6, after writing the vocabulary vector represented by the interval number into the target translation model, the method further comprises: The vocabulary vectors included in the distribution intervals outside the quantization interval are respectively written into the target translation model with corresponding original values, and are uniformly marked with a special number N.

8. The translation model compression method according to any one of claims 1 to 4, wherein the quantization interval is a symmetrical distribution interval or an asymmetrical distribution interval.

9. The translation model compression method according to any one of claims 1 to 4, after writing the vocabulary vector represented by the interval number into the target translation model, the method further comprises: The correspondence between each quantization interval segment and the assigned interval number is saved, wherein the quantization interval segment is a vocabulary vector distribution interval with the original value corresponding to the element of the vocabulary vector as the boundary.

10. A translation method comprising: Determine the target original text to be translated; Based on the target original text, searching the target translation model for a vocabulary vector corresponding to each vocabulary in at least one vocabulary contained in the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector quantized by the translation model compression method according to any one of claims 1 to 9; According to the correspondence between the vocabulary vector and the locally stored relationship, inverse quantization is used to determine the final value of each element in the vocabulary vector, wherein the correspondence is pre-stored by the translation model compression method according to claim 9, and the final value of each element in the vocabulary vector determined by inverse quantization is the same as the bit position of the original value; The final value determined by dequantization of each vocabulary included in the target original text is input into the target translation model for inference prediction, and the translation result is output.

11. The translation method according to claim 10, wherein the step of inverse quantizing and determining the final value corresponding to each element in the vocabulary vector according to the correspondence between the vocabulary vector and the locally stored data comprises: If the word list vector is represented by an interval number, then based on the word list vector, from the locally stored correspondence relationship, a quantization interval segment matching the interval number corresponding to each element in the word list vector is searched; and a node value in the quantization interval segment that meets a preset rule is determined as the final value of the element corresponding to the interval number; If the vocabulary vector is marked with a special number, the original value of each element in the vocabulary vector is directly determined as the final value of each element in the vocabulary vector. 12 . The translation method according to claim 11 , wherein the node value that complies with the preset rule is the maximum boundary value, the minimum boundary value, or the median of the quantization interval segment.

13. A translation model compression device, comprising: An extraction module extracts a vocabulary vector set from the trained target translation model, wherein the elements of the vocabulary vectors in the vocabulary vector set are normally distributed; A determination module, which determines a quantization interval based on a normal distribution result of the vocabulary vector set mapping; A division module, which calculates the number of interval segments to be divided according to the determined quantization accuracy, divides the quantization interval into a plurality of quantization interval segments having the same number as the interval segments in an equal manner, and assigns a unique interval number to each quantization interval segment; A quantization module replaces the original values ​​corresponding to the elements of the vocabulary vector contained in each quantization interval with the interval number of the corresponding quantization interval, and writes the vocabulary vector represented by the interval number into the target translation model; The bit position of the interval number is smaller than the bit position of the original value.

14. A translation device, comprising: The first determination module determines the target original text to be translated; A search module, based on the target original text, searches the target translation model for a vocabulary vector corresponding to each vocabulary in at least one vocabulary contained in the target original text, wherein the vocabulary vector stored in the target translation model is a vocabulary vector represented by an interval number after quantization processing using the translation model compression method according to any one of claims 1 to 9; A second determination module, according to the correspondence between the vocabulary vector and the locally stored vector, inverse quantization determines the final value corresponding to each element in the vocabulary vector, wherein the vector correspondence is pre-stored by the translation model compression method according to claim 9, and the final value of each element in the vocabulary vector determined by inverse quantization is the same as the bit position of the original value; The translation module inputs the final value determined by dequantization of each vocabulary included in the target original text into the target translation model for inference prediction, and outputs the translation result.

15. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the translation model compression method according to any one of claims 1 to 9, or the translation method according to any one of claims 10 to 12.

16. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the translation model compression method according to any one of claims 1 to 9, or the translation method according to any one of claims 10 to 12.

Images