Speech recognition method, hardware calling method and related device, processor and medium

Abstract

The application discloses a speech recognition method, a hardware calling method, related devices, a processor and a medium. The speech recognition method comprises the following steps: inputting the acoustic features of the audio frames in the speech to be recognized into an algorithm scheme for speech recognition; in response to the target operator to be currently executed in the algorithm scheme being a layer normalization operator, calling the integer output values of each output node in the previous operator of the target operator and the second quantization parameters of the factors in the normalization operator from the on-chip memory; calling the on-chip adder, the on-chip multiplier, the on-chip comparator, the on-chip shifter, the mapping relationship and the reciprocal square root table to process each integer output value and each quantization parameter, so as to obtain the layer normalization values of the integer output values; and sending the layer normalization values of each integer output value to the next operator of the target operator until the algorithm scheme is executed completely, so as to obtain the recognition text of the speech to be recognized. The above scheme can improve the operation efficiency of the algorithm scheme and the applicability of the algorithm scheme on various processors.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer hardware, and particularly relates to a speech recognition method, a hardware calling method and related devices, a processor and a medium. BACKGROUND

[0002] With the rapid development of electronic information technology, algorithm schemes have been deployed in numerous intelligent hardware to provide intelligent assistance for fields such as smart home, smart city and smart production.

[0003] Taking speech recognition as an example, layer normalization (LN) is usually contained in an algorithm scheme. For example, a recognition model for speech such as a recurrent neural network or an encoder-decoder usually contains layer normalization. At present, normalization calculation mostly adopts floating-point or half-floating-point calculation methods, which are low in efficiency and even not supported by embedded processors. In addition, because the floating-point operation precision of different processors is different, when the algorithm scheme is implemented by using a neural network, the calculation results in the training and testing stages (usually using a GPU) and the calculation results in the engine inference stage (usually using a CPU or NPU) cannot be completely consistent, which makes it difficult to determine whether the precision loss of the algorithm scheme is caused by calculation errors or network problems. Therefore, how to improve the operation efficiency of the algorithm scheme and its applicability on various processors, and reduce the complexity of problem positioning when the algorithm scheme uses a neural network, has become a problem to be solved. SUMMARY

[0004] The present application mainly solves the technical problem of providing a speech recognition method, a hardware calling method and related devices, a processor and a medium, which can improve the operation efficiency of the algorithm scheme and its applicability on various processors, and reduce the complexity of problem positioning when the algorithm scheme uses a neural network.

[0005] To solve the above technical problems, the first aspect of the present application provides a speech recognition method, comprising: obtaining a first quantization parameter set by a user for a square root reciprocal operation in a preset numerical value range, and obtaining a square root reciprocal table quantized by the first quantization parameter and a mapping relationship between numerical values in the preset numerical value range and order positions in the square root reciprocal table; inputting acoustic features of an audio frame in the speech to be recognized into an algorithm scheme for speech recognition; in response to a target operator currently to be executed in the algorithm scheme being a layer normalization operator, calling from an on-chip memory integer output values of each output node in a previous operator of the target operator and a second quantization parameter of a factor in a normalization operator; calling an on-chip adder, an on-chip multiplier, an on-chip comparator, an on-chip shifter, and the mapping relationship and the square root reciprocal table to process each integer output value and each quantization parameter, to obtain a layer normalization value of the integer output value; and sending the layer normalization value of each integer output value to a next operator of the target operator, until the algorithm scheme is executed completely, to obtain a recognition text of the speech to be recognized.

[0006] To solve the above technical problems, the second aspect of the present application provides a hardware calling method, comprising: obtaining a first quantization parameter set by a user for a square root reciprocal operation in a preset numerical value range, and obtaining a square root reciprocal table quantized by the first quantization parameter and a mapping relationship between numerical values in the preset numerical value range and order positions in the square root reciprocal table; in response to a target operator currently to be executed in the algorithm scheme being a layer normalization operator, calling from an on-chip memory integer output values of each output node in a previous operator of the target operator and a second quantization parameter of a factor in a normalization operator; calling an on-chip adder, an on-chip multiplier, an on-chip comparator, an on-chip shifter, and the mapping relationship and the square root reciprocal table to process each integer output value and each quantization parameter, to obtain a layer normalization value of the integer output value.

[0007] To solve the above technical problems, the third aspect of the present application provides a speech recognition device, comprising: an acquisition module, an input module, a calling module, a calling module, and an output module, the acquisition module is used for acquiring a first quantization parameter set by a user for a square root reciprocal operation in a preset numerical value range, and acquiring a square root reciprocal table quantized by the first quantization parameter and a mapping relationship between values in the preset numerical value range and order bits in the square root reciprocal table; the input module is used for inputting the acoustic features of the audio frame in the speech to be recognized into an algorithm scheme for speech recognition; the calling module is used for calling the integer output values of each output node in the previous operator of the target operator and the second quantization parameter of the factor in the normalization operator from the on-chip memory in response to the target operator currently to be executed in the algorithm scheme being a layer normalization operator; the calling module is used for calling the on-chip adder, the on-chip multiplier, the on-chip comparator, the on-chip shifter, and the mapping relationship and the square root reciprocal table to process each integer output value and each quantization parameter to obtain the layer normalization value of the integer output value; and the output module is used for sending the layer normalization value of each integer output value into the next operator of the target operator until the algorithm scheme is executed to obtain the recognition text of the speech to be recognized.

[0008] To solve the above technical problems, the fourth aspect of the present application provides a hardware calling device, comprising: an acquisition module, a calling module, and a calling module, the acquisition module is used for acquiring a first quantization parameter set by a user for a square root reciprocal operation in a preset numerical value range, and acquiring a square root reciprocal table quantized by the first quantization parameter and a mapping relationship between values in the preset numerical value range and order bits in the square root reciprocal table; the calling module is used for calling the integer output values of each output node in the previous operator of the target operator and the second quantization parameter of the factor in the normalization operator from the on-chip memory in response to the target operator currently to be executed in the algorithm scheme being a layer normalization operator; the calling module is used for calling the on-chip adder, the on-chip multiplier, the on-chip comparator, the on-chip shifter, and the mapping relationship and the square root reciprocal table to process each integer output value and each quantization parameter to obtain the layer normalization value of the integer output value.

[0009] To solve the above technical problems, the fifth aspect of the present application provides a processor, comprising an operation scheduler, and an on-chip memory, an on-chip adder, an on-chip multiplier, an on-chip shifter, and an on-chip comparator connected with the operation scheduler, respectively, the operation scheduler is used for executing program instructions to realize the speech recognition method in the first aspect or realize the hardware calling method in the second aspect.

[0010] To solve the above technical problems, the sixth aspect of the present application provides a computer readable storage medium, which stores program instructions capable of being executed by a processor, the program instructions are used for realizing the speech recognition method of the first aspect or realizing the hardware calling method of the second aspect.

[0011] The above scheme acquires the first quantization parameter set by the user for the reciprocal square root operation in the preset numerical range, inputs the acoustic features of the audio frame in the to-be-recognized speech into the algorithm scheme for speech recognition through the reciprocal square root table quantized by the first quantization parameter and the mapping relationship between the numerical values in the preset numerical range and the sequence positions in the reciprocal square root table, and obtains the layer normalization value of the integer output value by calling the in-chip adder, the in-chip multiplier, the in-chip comparator, the in-chip shifter, the mapping relationship and the reciprocal square root table to process each integer output value and each quantization parameter, in response to the target operator currently to be executed in the algorithm scheme being a layer normalization operator, calling the second quantization parameter of the factor in the normalization operator and the integer output value of each output node in the previous operator of the target operator from the in-chip memory, and then sending the layer normalization value of each integer output value to the next operator of the target operator until the algorithm scheme is executed completely to obtain the recognized text of the to-be-recognized speech. Therefore, by pre-acquiring the mapping relationship between the numerical values in the preset numerical range and the sequence positions in the reciprocal square root table and the quantized reciprocal square root table, when the layer normalization operator is executed, only fixed-point calculation needs to be performed by combining the in-chip memory, the in-chip adder, the in-chip multiplier, the in-chip shifter and the in-chip comparator, without floating-point calculation, so that the operation efficiency of the algorithm scheme can be improved, the applicability of the algorithm scheme on various processors can be improved, and the complexity of problem positioning can be reduced when the algorithm scheme adopts a neural network. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a flowchart of an embodiment of the speech recognition method of the present application;

[0013] Figure 2 is a schematic diagram of an embodiment of the reciprocal square root table;

[0014] Figure 3 is a continuation of the reciprocal square root table shown in Figure 2

[0015] Figure 4 is a schematic diagram of the framework of an embodiment of the algorithm scheme;

[0016] Figure 5 is a flowchart of an embodiment of the hardware calling method;

[0017] Figure 6 is a schematic diagram of the framework of an embodiment of the speech recognition device of the present application;

[0018] Figure 7 is a schematic diagram of the framework of an embodiment of the hardware calling device of the present application;

[0019] Figure 8 is a schematic diagram of the framework of an embodiment of the processor of the present application;

[0020] Figure 9 ​is a schematic diagram of a framework of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0021] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0022] In the following description, specific details are set forth in connection with the

[0023] The terms "system" and "network" are often used interchangeably herein. The term "and / or" herein is merely used to represent an associated relationship between associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" herein means two or more than two.

[0024] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the voice recognition method of the present application.

[0025] Specifically, it can include the following steps:

[0026] Step S11: Obtain the first quantization parameter set by the user for the inverse square root operation in the preset numerical range, and obtain the mapping relationship between the numerical values in the preset numerical range and the sequence positions in the inverse square root table after the first quantization parameter quantization.

[0027] In one implementation scenario, considering that the inverse square root table is loaded into the on-chip memory (such as register group, etc.) of the processor when the algorithm scheme runs, the first quantization parameter can be specifically set according to the storage space of the on-chip register. Specifically, the larger the storage space of the on-chip register, the larger the first quantization parameter can be set, and vice versa, the smaller the storage space of the on-chip register, the smaller the first quantization parameter can be set. For example, taking the output precision of the inverse square root table as an unsigned 16-bit integer value, the first quantization parameter can be set to 14. Other cases can be similarly extended, which will not be repeated here.

[0028] In one implementation scenario, the preset numerical range can be set according to actual conditions. For example, in order to improve the accuracy as much as possible, the preset numerical range can be preferably set to [0.25, 1) and the like. It should be noted that the square brackets represent the endpoint value, while the round brackets represent the endpoint value. Of course, in the case of relatively loose accuracy requirements, the preset numerical range can also be set to other conditions, which are not limited here.

[0029] In the embodiments of the present disclosure, the mapping relationship and the reciprocal square root table are loaded into the on-chip memory at the runtime of the algorithm scheme. It should be noted that the above-mentioned preset numerical range and the first quantization parameter can be pre-set by a user, and the mapping relationship and the reciprocal square root table can be obtained according to the preset numerical range and the first quantization parameter on an operation platform supporting floating point operation. The mapping relationship and the reciprocal square root table obtained are loaded into the on-chip memory of the processor running the algorithm scheme at the runtime of the algorithm scheme, so that the mapping relationship and the reciprocal square root table can be quickly called during the running of the algorithm scheme.

[0030] In one implementation scenario, the algorithm scheme can include a plurality of operators connected in sequence, and each operator can include at least one input node and at least one output node.

[0031] In one specific implementation scenario, the operators in the algorithm scheme can be implemented by a function expression. For example, for an algorithm scheme for image optimization, it can include a pre-processing function (such as cropping, rigid transformation, etc.), a Gaussian kernel function, etc. connected in sequence. Other cases can be similarly extended, and will not be exemplified one by one here.

[0032] In one specific implementation scenario, the operators in the algorithm scheme can also be implemented by a neural network. For example, for an algorithm scheme for image classification, it can include a convolution layer, a pooling layer, a fully connected layer, etc. connected in sequence. Other cases can be similarly extended, and will not be exemplified one by one here.

[0033] In one implementation scenario, the mapping relationship can be represented as an integer of a product of a difference between any value in a preset value range and a preset value and a preset value range, and the preset value is a ratio of a number table capacity of the reciprocal square root table divided by a difference between an upper limit value and a lower limit value of the preset value range. The number table capacity represents a total number of values available for querying in the reciprocal square root table. The number table capacity can be set according to a storage space of the on-chip memory. Similarly to the first quantization parameter, the larger the storage space of the on-chip memory, the larger the number table capacity can be set, and vice versa. In addition, the higher the accuracy requirement for querying the reciprocal square root table, the larger the number table capacity can be set, and the lower the accuracy requirement for querying the reciprocal square root table, the smaller the number table capacity can be set. For example, when the number table capacity is set to 768 (i.e., the reciprocal square root table contains 768 values available for querying), and the preset value range is set to [0.25, 1), the original number difference between adjacent values in the reciprocal square root table is (1-0.25) / 768 = 0.0009765625. On this basis, for any value n in the preset value range, according to the mapping relationship, the reciprocal square root of n can be directly queried in the reciprocal square root table according to the following sequence:

[0034]

[0035] In the above formula (1), round represents an integer operation. In addition, when the preset value range and the number table capacity are set to other conditions, the mapping relationship can be similarly derived, which will not be repeated one by one. In the above manner, the mapping relationship is represented as an integer of a product of a difference between any value in a preset value range and a preset value and a preset value range, and the preset value is a ratio of a number table capacity of the reciprocal square root table divided by a difference between an upper limit value and a lower limit value of the preset value range, which improves the accuracy of querying the reciprocal square root table.

[0036] In one implementation scenario, in order to obtain the square root reciprocal table quantized by the first quantization parameter, the third product of the reciprocal of the preset value and the sequence position can be obtained first, and the second sum of the third product and the lower limit value can be obtained. On this basis, the integer of the square root reciprocal of the second sum quantized by the first quantization parameter can be obtained as the value of the square root reciprocal table at the sequence position. It should be noted that, in order to facilitate subsequent processor operation, in the disclosed embodiments of the present application, the quantization can be performed with 2 as the base and the quantization parameter as the power, so that the processor can directly perform left shift and right shift through the on-chip shifter to realize related operations. Still taking the preset value range set as [0.25, 1) and the table capacity set as 768 as an example, the reciprocal of the preset value is 768 / (1-0.25)=1024. For sequence position 1, the third product is 1024, and the second sum of the third product and the lower limit value is 1024.25, so the integer of the square root reciprocal of the second sum quantized by 2 14 The quantized value is 32768. Please refer to Figure 2 and Figure 3 , Figure 2 is a schematic diagram of an embodiment of the square root reciprocal table, Figure 3 is Figure 2 a continuation of the square root reciprocal table shown in FIG. 8. The values of sequence position 2 to sequence position 768 in the quantized square root reciprocal table can be obtained by the foregoing method respectively, which will not be exemplified one by one here. The above-mentioned method of obtaining the third product of the reciprocal of the preset value and the sequence position, obtaining the second sum of the third product and the lower limit value, and then obtaining the integer of the square root reciprocal of the second sum quantized by the first quantization parameter as the value of the square root reciprocal table at the sequence position can improve the accuracy of the square root reciprocal table.

[0037] It should be noted that the quantized square root reciprocal table can be stored in the on-chip memory in binary.

[0038] Step S12: input the acoustic feature of the audio frame in the to-be-identified voice into the algorithm scheme for voice recognition.

[0039] It should be noted that the acoustic feature can include but is not limited to Fbank, MFCC, etc., which is not limited here. In addition, the algorithm scheme runs on a processor chip. The kernel of the current conventional AI chip usually adopts a heterogeneous architecture of ARM+DSP+NPU. Among them, large-scale calculation is usually completed by DSP+NPU, and NPU is suitable for large-scale fixed-point operation, and there is no dedicated floating-point operation unit. If the prior art is adopted, the deployment requirements cannot be met on such a processor, but by adopting the disclosed embodiments of the present application, the deployment can be completed on such a processor without a dedicated floating-point operation unit or with weak floating-point operation unit processing capability, especially an embedded processor.

[0040] Specifically, as mentioned before, the algorithm scheme can be implemented by a neural network. Please refer to Figure 4 , Figure 4 is a schematic diagram of a framework of an embodiment of the algorithm scheme. More specifically, Figure 4 is a schematic diagram of a framework of an embodiment of the algorithm scheme for speech recognition. As shown in Figure 4 , the algorithm scheme can include a Transformer structure. Figure 4 The specific meanings of the operators in the algorithm scheme shown in Figure 4 are described in detail in the technical details of a network model including a Transformer structure, such as an encoder-decoder (i.e., Encoder-Decoder), which will not be described here. It should be noted that

[0041] Step S13: In response to the target operator currently to be executed in the algorithm scheme being a layer normalization operator, the integer output values of each output node in the previous operator of the target operator and the second quantization parameters of the factors in the normalization operator are retrieved from the on-chip memory.

[0042] In one implementation scenario, the integer output values of the output nodes can be int8 (i.e., 8-bit integer), and of course, can also be set to 16-bit integer, etc. according to actual application needs, which will not be limited here. In addition, for ease of description, the total number of each output node can be denoted as N, and the integer output value of the i-th output node can be denoted as x i .

[0043] In one implementation scenario, the factors in the layer normalization operator can include a scaling factor and a translation factor, for ease of description, the scaling factor can be denoted as γ, and the translation factor can be denoted as β, the specific meanings of the scaling factor and the translation factor can be referred to the technical details of layer normalization, which will not be described here. Further, the second quantization parameter of the scaling factor γ can be denoted as q γ , and the second quantization parameter of the translation factor β can be denoted as q β .

[0044] In one specific implementation scenario, the second quantization parameters of the scaling factor and the translation factor can be set by the user, such as 10, 11, 12, etc., which will not be limited here.

[0045] In one specific implementation scenario, as mentioned above, the algorithm scheme can be implemented by a neural network. In this case, the algorithm scheme can be trained and tested in advance before inference application. In the training stage, a pseudo-quantization operation can be performed on the operators in the algorithm scheme to simulate the calculation loss, and the quantization parameters involved in the layer normalization and other operators are adjusted according to the loss until the training converges, that is, the quantization parameters at this time are saved as the second quantization parameters. The specific process of the pseudo-quantization operation can be referred to the related description below, which is not described here in detail.

[0046] Step S14: calling the on-chip adder, on-chip multiplier, on-chip comparator, on-chip shifter, and mapping relationship and square root reciprocal table to process each integer output value and each quantization parameter to obtain the layer normalization value of the integer output value.

[0047] Specifically, the first difference value between the product of the integer output value and the total number of nodes and the cumulative sum of the integer output value, and the second difference value between the product of the total number of nodes and the square cumulative sum of the integer output value and the square of the cumulative sum can be obtained by calling the on-chip adder and on-chip multiplier, the second difference value is quantized by the third quantization parameter in the first value in the preset value range by calling the on-chip comparator, and the quantized value of the square root reciprocal of the first value is found as the second value in the square root reciprocal table by calling the mapping relationship, and finally the layer normalization value of the integer output value is obtained by calling the on-chip multiplier and on-chip shifter to process the first difference value, the second value and each quantization parameter. It should be noted that the total number of nodes represents the total number of output nodes in the previous operator.

[0048] In one implementation scenario, the cumulative sum of each integer output value can be obtained by calling the on-chip adder, and the square cumulative sum of each integer output value can be obtained by calling the on-chip adder and on-chip multiplier, and the first difference value between the product of the integer output value and the total number of nodes and the cumulative sum, and the second difference value between the product of the total number of nodes and the square cumulative sum and the square of the cumulative sum can be obtained by calling the on-chip multiplier and on-chip adder.

[0049] In one specific implementation scenario, the cumulative sum of each integer output value can be obtained by calling the on-chip adder, and for the sake of description, the cumulative sum can be denoted as In addition, the square of each integer output value can be calculated by calling the on-chip multiplier, and the square cumulative sum of each integer output value can be calculated by calling the on-chip adder, and for the sake of description, the square cumulative sum can be denoted as

[0050] In one specific implementation scenario, for each integer output value that needs to be processed by layer normalization, the product of the integer output value and the total number of output nodes N can be calculated by calling the multiplier, and for the sake of description, it can be denoted as N x x. On this basis, the first difference value with the cumulative sum can be calculated by calling the adder, and for the sake of description, it can be denoted as

[0051] In one specific implementation scenario, a multiplier can be invoked to calculate the product of the total number N and the square accumulation sum, which can be denoted as A multiplier can be further invoked to calculate the square of the accumulation sum, which can be denoted as On this basis, an adder can be further invoked to calculate the second difference between the two, which can be denoted as

[0052] In one implementation scenario, a comparator can be invoked to split the second difference The first value in the preset value range is quantized by a third quantization parameter. Since the first value needs to be queried by the reciprocal of square root table subsequently, the base of the third quantization parameter can be 4. For ease of description, the first value can be denoted as n', and the second value can be split by the comparator as n' x 4 q1 , where q1 represents the third quantization parameter, and n' is located in the preset value range (for example, the value range of 0.25 to 1 mentioned above).

[0053] In one implementation scenario, after the comparator is invoked to obtain the first value n', a mapping relationship can be further invoked to obtain the target sequence position corresponding to the first value n'. For example, in the case where the table capacity is set to 768 and the preset value range is set to [0.25, 1), the target sequence position key corresponding to the first value n' can be obtained by the mapping relationship shown in formula (1), and the quantized reciprocal of square root table shown in formula (2) and formula (3) can be invoked to find the second value located in the target sequence position key. Figure 2 and 3 Since the second value is quantized by the reciprocal of square root by the first quantization parameter 14, it can be represented as When the first quantization parameter is set to other values, the same can be applied, which will not be exemplified one by one.

[0054] It should be noted that since the second difference is split as n' x 4 q1 , the first value n' can actually be represented as:

[0055]

[0056] And since the second value is represented as , the above formula (2) can be substituted into the expression to obtain that the second value can actually be represented as:

[0057]

[0058] Further, to ensure that the processor can smoothly calculate the second value without crashing and reporting errors, when the aforementioned on-chip multiplier and on-chip adder are called to calculate the second difference value, the calculated second difference value on the basis of the minimum value ε. That is, at this time, the second value can actually be represented as:

[0059]

[0060] In one implementation scenario, in the process of calling the on-chip multiplier and on-chip shifter to process the first difference value, the second value, and each quantization parameter to obtain the layer normalization value of the integer output value, the on-chip multiplier can be first called to obtain the first product of the first difference value and the second value. For ease of description, the first product can be denoted as:

[0061]

[0062] and the on-chip shifter is called to obtain the third value of the first product left shifted by the quantization parameter (denoted as q x ) of the integer output value and right shifted by the first quantization parameter (such as the aforementioned 14) and the third quantization parameter (such as the aforementioned q1):

[0063]

[0064] In the aforementioned formula (6), that is, the on-chip shifter is called to left shift the first product by the quantization parameter of the integer output value, and that is, the on-chip shifter is further called to right shift by the first quantization parameter and the third quantization parameter, and finally the third value can be represented as:

[0065]

[0066] On this basis, the on-chip multiplier can be called to obtain the second product of the third value and the scaling factor γ:

[0067]

[0068] and the on-chip shifter is continued to be called to obtain the fourth value of the second product left shifted by the second quantization parameter (such as the aforementioned q γ ) of the scaling factor:

[0069]

[0070] and the on-chip shifter is called to obtain the fifth value of the translation factor β left shifted by the quantization parameter (such as the aforementioned q x ) of the integer output value and the second quantization parameter (such as the aforementioned q γ ) of the scaling factor:

[0071]

[0072] Further, the in-slice adder can be invoked to obtain the first sum of the fourth value and the fifth value:

[0073]

[0074] On this basis, the in-slice shifter can be invoked to left shift the first sum by the quantization parameter of the layer normalization value (denoted as q o ), and right shift the first sum by the quantization parameter of the scaling factor (q γ ), and the quantization parameter of the integer output value (q x ) respectively:

[0075]

[0076] In the above formula (12), that is, the in-slice shifter is invoked to left shift the first sum by the quantization parameter of the layer normalization value q o , and right shift the first sum by the quantization parameter of the scaling factor q γ , and the quantization parameter of the integer output value q x , the above formula (12) can be further simplified as:

[0077]

[0078] In the above formula (13), the numerator and the denominator of the fraction are simultaneously divided by N, and the above formula (13) can be further simplified as:

[0079]

[0080] In the above formula (14), that is, the average value of the integer output values of each output node, which can be denoted as E(x). It should be noted that since ε itself is a minimum value, dividing it by any integer greater than 1 is still a minimum value, so the denominator in formula (14) is still denoted as ε. Therefore, the above formula (14) can be further simplified as:

[0081]

[0082] Further, in the above formula (15), can be expanded as: Therefore, the above formula (15) can be further transformed as:

[0083]

[0084] In the above formula (16), can be transformed as that is, can be transformed as i.e. N x (E(x)) 2 Therefore, formula (16) can be further transformed as:

[0085]

[0086] Further transformation can be:

[0087]

[0088] Further transformation can be:

[0089]

[0090] In the above formula (19), i.e. variance, which can be denoted as var(x), and the final output value can be further transformed as:

[0091]

[0092] The value shown in the above formula (20) is the value of the integer output value x after layer normalization. That is, the value obtained by the above formula (12) is essentially the value of the integer output value x after layer normalization.

[0093] It should be noted that when the algorithm scheme is implemented by a neural network, in the forward operation process, as mentioned above, the cumulative sum and squared cumulative sum of the integer output values of each output node in the previous operator can be obtained, and substituted into formula (13), and the value range of formula (13) is calculated, and the quantization parameter q x of the integer output value is obtained according to the Q value quantization method. The specific calculation process can be referred to in the following related description, which is not described here. Similarly, the value range of the scaling factor γ and the translation factor β can be calculated, and the quantization parameters q γ and q β of the two can be obtained according to the Q value quantization method. On this basis, these Q values can be used for pseudo-quantization operation, and the network parameters are updated by backward iteration. In addition, in the training process, the above Q values can be adjusted with the network parameters of the algorithm scheme until the network converges, and the converged Q values can be taken as the final quantization parameters for saving.

[0094] In one implementation scenario, please refer to the following quantization formula:

[0095] r = S(q - Z) …… (21)

[0096]

[0097] The above formula (22) can be converted from the above formula (21), where r represents a floating-point real number, and q represents a quantized fixed-point integer. S and Z both represent quantization parameters, S is a scale representing the proportional relationship between the floating-point real number and the fixed-point integer, and Z is a zero point representing the fixed-point integer corresponding to 0 in the floating-point real number after quantization. The calculation methods of the two are as follows:

[0098]

[0099]

[0100] In the above formulas (23) and (24), r max and r min respectively represent the maximum floating-point real number and the minimum floating-point real number obtained by statistics, q max and q min respectively represent the maximum value and the minimum value that the fixed-point integer can represent. In addition, when Z is 0, the quantization operation is symmetric quantization, and when Z is not 0, the quantization operation is asymmetric quantization. Further, as described above, S can be expressed as a power of 2, which is Q value quantization. In addition, when S is not expressed as a power of 2, it is maximum value quantization. In the embodiments disclosed in the present application, the symmetric Q value quantization method can be used, that is, S = 2 q , and 2 q-1 ≤max(abs(r))≤2 q , where abs represents the absolute value.

[0101] In one implementation scenario, Quantization Aware Training is to insert a fake quantization module in the model to simulate the rounding and clamping operations of the quantization model in the inference process, so as to improve the adaptability of the model to the quantization effect in the training process and obtain higher precision of the quantization model. In this process, all calculations (including model forward and backward propagation calculations and fake quantization node calculations) are implemented by floating-point calculation, and the real int8 model is quantized only after the training is completed, as follows:

[0102] y = round(x * scale) / scale …… (25)

[0103] In other words, the original floating-point real number can be quantized (i.e., multiplied by the scale and then rounded), and then divided by the scale to simulate quantization loss. Actual testing shows that although this method sacrifices some data precision, compared to existing technologies, the overall performance of the algorithm remains controllable because the training and inference processes can be kept consistent. It also reduces the number of parameters and improves computational efficiency. Furthermore, it can be widely applied to as many computing platforms as possible, especially on hardware platforms with lower-performance embedded CPUs.

[0104] Step S15: Feed the layer normalized value of each integer output value into the next operator of the target operator until the algorithm scheme is completed, and obtain the recognized text of the speech to be recognized.

[0105] Specifically, with Figure 4 Taking the algorithm scheme shown as an example, after all operators in the algorithm scheme have been processed, the decoding probability of each character in the preset dictionary as the decoding character at the current decoding time t can be obtained. Then, several characters can be selected as decoding characters according to the decoding probability, and the decoding process is repeated until the decoding probability of the ending character is maximized at decoding time t+N. The decoding can then be considered to be over. Thus, the beam search strategy can be used to obtain the recognition text of the speech to be recognized based on the decoding characters obtained at each decoding time.

[0106] The above scheme obtains the first quantization parameter set by the user for the square root reciprocal operation within a preset numerical range, and obtains the square root reciprocal table quantized by the first quantization parameter and the mapping relationship between the numerical values ​​within the preset numerical range and the ordinal positions in the square root reciprocal table. The acoustic features of the audio frames in the speech to be recognized are input into the speech recognition algorithm. In response to the target operator being a layer normalization operator in the algorithm, the scheme retrieves the integer output values ​​of each output node from the previous operator of the target operator and the second quantization parameter of the factor in the normalization operator from the on-chip memory. Then, it calls the on-chip adder, on-chip multiplier, on-chip comparator, on-chip shifter, and the mapping relationship and square root reciprocal table to process each integer output. The algorithm obtains the layer normalized value of the integer output value by taking the output value and various quantization parameters. Then, the layer normalized value of each integer output value is sent to the next operator of the target operator until the algorithm is completed and the recognized text of the speech to be recognized is obtained. Therefore, by obtaining the mapping relationship between the numerical values ​​within the preset range and the ordinal position in the square root reciprocal table and the quantized square root reciprocal table in advance, when executing the layer normalization operator, only fixed-point calculations need to be performed by combining on-chip memory, on-chip adder, on-chip multiplier, on-chip shifter and on-chip comparator, without the need for floating-point calculations. Therefore, the computational efficiency of the algorithm can be improved, as well as its applicability on various processors. When the algorithm adopts a neural network, the complexity of problem localization is reduced.

[0107] Please seeFigure 5 , Figure 5 is a flowchart of an embodiment of the hardware calling method. Specifically, the method can include the following steps:

[0108] Step S51: Obtain a first quantization parameter set by a user for a square root reciprocal operation in a preset numerical range, and obtain a square root reciprocal table quantized by the first quantization parameter and a mapping relationship between numerical values in the preset numerical range and the order bits in the square root reciprocal table.

[0109] In the embodiment of the present disclosure, the mapping relationship and the square root reciprocal table are loaded into an on-chip memory at runtime of the algorithm scheme. It should be noted that the related meanings of the first quantization parameter, the mapping relationship, and the square root reciprocal table can be referred to the related descriptions in the foregoing disclosed embodiments, which will not be described here.

[0110] Step S52: In response to the target operator currently to be executed in the algorithm scheme being a layer normalization operator, call the integer output values of each output node in the previous operator of the target operator and the second quantization parameter of the factor in the normalization operator.

[0111] Specifically, the related descriptions can be referred to the foregoing disclosed embodiments, which will not be described here.

[0112] Step S53: Call the on-chip adder, the on-chip multiplier, the on-chip comparator, the on-chip shifter, and the mapping relationship and the square root reciprocal table to process each integer output value and each quantization parameter to obtain a layer normalization value of the integer output value.

[0113] Specifically, the related descriptions can be referred to the foregoing disclosed embodiments, which will not be described here.

[0114] The above scheme, by pre-obtaining the mapping relationship between numerical values in a preset numerical range and order bits in a square root reciprocal table and the quantized square root reciprocal table, when running a layer normalization network layer, only fixed-point calculation needs to be done in combination with the on-chip memory, the on-chip adder, the on-chip multiplier, the on-chip shifter, and the on-chip comparator, without floating-point calculation, so as to improve the operation efficiency of the algorithm scheme and the applicability on various processors, and when the algorithm scheme adopts a neural network, reduce the complexity of problem positioning.

[0115] It should be noted that although the above disclosed embodiments are described as applying to speech recognition to improve the operation efficiency of speech recognition and the applicability of speech recognition on various processors, the steps in the hardware calling method embodiments of the present application are not limited to only being applied to speech recognition. For example, the algorithm scheme can also be adaptively adjusted according to actual application needs to be implemented in other scenarios such as translation, reading comprehension, etc. that require layer normalization, thereby improving the operation efficiency of translation, reading comprehension, etc. and the applicability on various processors.

[0116] Please refer to Figure 6 , Figure 6 is a framework schematic diagram of an embodiment of the hardware calling device 60 of the present application. The hardware calling device 60 comprises an acquisition module 61, an input module 62, a calling module 63, a calling module 64, and an output module 65. The acquisition module 61 is configured to acquire a first quantization parameter set by a user for a square root reciprocal operation in a preset value range, and acquire a square root reciprocal table quantized by the first quantization parameter and a mapping relationship between values in the preset value range and order positions in the square root reciprocal table; the input module 62 is configured to input acoustic features of an audio frame in the speech to be recognized into an algorithm scheme for speech recognition; the calling module 63 is configured to, in response to a target operator currently to be executed in the algorithm scheme being a layer normalization operator, call integer output values of each output node in a previous operator of the target operator and a second quantization parameter of a factor in the normalization operator; the calling module 64 is configured to call an on-chip adder, an on-chip multiplier, an on-chip comparator, an on-chip shifter, and the mapping relationship and the square root reciprocal table to process each integer output value and each quantization parameter to obtain a layer normalization value of the integer output value; and the output module 65 is configured to send the layer normalization value of each integer output value into a next operator of the target operator until the algorithm scheme is executed to obtain a recognized text of the speech to be recognized.

[0117] The above scheme, the speech recognition device 60 acquires the mapping relationship between values in the preset value range and order positions in the square root reciprocal table and the quantized square root reciprocal table in advance, and when running the layer normalization network layer, only needs to combine the on-chip memory, the on-chip adder, the on-chip multiplier, the on-chip shifter, and the on-chip comparator to do fixed-point calculation, without doing floating-point calculation, so as to improve the operation efficiency of the algorithm scheme and the applicability on various processors, and when the algorithm scheme adopts a neural network, reduce the complexity of problem positioning.

[0118] In some disclosed embodiments, the calling module 64 comprises a first calling sub-module for calling the in-chip adder, the in-chip multiplier, the product of the integral output value and the total number of nodes, the first difference between the integral output value and the cumulative sum, and the second difference between the total number of nodes and the square cumulative sum of the integral output value and the square of the cumulative sum; the calling module 64 comprises a second calling sub-module for calling the in-chip comparator to split the second difference into a first value in a preset value range and quantize the first value by a third quantization parameter, and calling the mapping relationship to find the quantized value of the square root reciprocal of the first value as a second value in the square root reciprocal table; the calling module 64 comprises a third calling sub-module for calling the in-chip multiplier and the in-chip shifter to process the first difference, the second value, and each quantization parameter to obtain the layer normalization value of the integral output value.

[0119] In some disclosed embodiments, the second calling sub-module comprises a sequence position acquisition unit for calling the mapping relationship to obtain the target sequence position corresponding to the first value; the second calling sub-module comprises a value searching unit for searching the second value in the square root reciprocal table at the target sequence position.

[0120] In some disclosed embodiments, the first calling sub-module comprises a cumulative sum acquisition unit for calling the in-chip adder to obtain the cumulative sum of each integral output value, the first calling sub-module comprises a square cumulative sum acquisition unit for calling the in-chip adder and the in-chip multiplier to obtain the square cumulative sum of each integral output value; the first calling sub-module comprises a first difference acquisition unit for calling the in-chip multiplier and the in-chip adder to obtain the first difference between the integral output value and the product of the total number of nodes and the cumulative sum, and the first calling sub-module comprises a second difference acquisition unit for obtaining the second difference between the product of the total number of nodes and the square cumulative sum and the square of the cumulative sum.

[0121] In some disclosed embodiments, the third calling sub-module comprises a first calling unit for calling the in-chip multiplier to obtain a first product of the first difference and the second value, and calling the in-chip shifter to obtain a third value of the first product left shifted by the quantization parameter of the integral output value and right shifted by the first quantization parameter and the third quantization parameter; the third calling sub-module comprises a second calling unit for calling the in-chip multiplier to obtain a second product of the third value and the scaling factor, and calling the in-chip shifter to obtain a fourth value of the second product left shifted by the second quantization parameter of the scaling factor, and calling the in-chip shifter to obtain a fifth value of the translation factor left shifted by the quantization parameter of the integral output value and the second quantization parameter of the scaling factor; the third calling sub-module comprises a third calling unit for calling the in-chip adder to obtain a first sum of the fourth value and the fifth value; the third calling sub-module comprises a fourth calling unit for calling the in-chip shifter to obtain the layer normalization value of the integral output value by left shifting the first sum by the quantization parameter of the layer normalization value and right shifting by the second quantization parameter of the scaling factor and the quantization parameter of the integral output value, respectively.

[0122] In some disclosed embodiments, the mapping relationship is represented as an integer of a product of a preset numerical value and a difference between a lower limit value of the preset numerical value range; wherein the preset numerical value is a ratio of a numerical table capacity of the reciprocal square root table divided by a difference between an upper limit value and the lower limit value of the preset numerical value range.

[0123] In some disclosed embodiments, the obtaining module 61 comprises a product obtaining sub-module for obtaining a third product of the reciprocal of the preset numerical value and the sequence position; the obtaining module 61 comprises a sum value obtaining sub-module for obtaining a second sum value of the third product and the lower limit value; the obtaining module 61 comprises an operation and integer obtaining sub-module for obtaining an integer of a value of the reciprocal square root after the first quantization parameter quantization as the numerical value of the reciprocal square root table at the sequence position.

[0124] In some disclosed embodiments, the algorithm scheme for speech recognition contains a Transformer structure.

[0125] Please refer to Figure 7 , Figure 7 is a framework schematic diagram of an embodiment of the hardware calling device 70 of the present application. The speech recognition device 70 comprises an obtaining module 71, a calling module 72 and a calling module 73. The obtaining module 71 is configured to obtain a first quantization parameter set by a user for a reciprocal square root operation in a preset numerical value range, and obtain a mapping relationship between a numerical value in the preset numerical value range and a sequence position in a reciprocal square root table after the first quantization parameter quantization; the calling module 72 is configured to, in response to a target operator currently to be executed in the algorithm scheme being a layer normalization operator, call an integer output value of each output node in a previous operator of the target operator and a second quantization parameter of a factor in the normalization operator; the calling module 73 is configured to call an on-chip adder, an on-chip multiplier, an on-chip comparator, an on-chip shifter and the mapping relationship and the reciprocal square root table to process each integer output value and each quantization parameter, so as to obtain a layer normalization value of the integer output value.

[0126] The above scheme, the hardware calling device 70 obtains the mapping relationship between the numerical value in the preset numerical value range and the sequence position in the reciprocal square root table and the quantized reciprocal square root table in advance, and when running the layer normalization network layer, only needs to combine the on-chip memory, the on-chip adder, the on-chip multiplier, the on-chip shifter and the on-chip comparator to do fixed-point calculation, without doing floating-point calculation, so as to improve the operation efficiency of the algorithm scheme and the applicability on various processors, and when the algorithm scheme adopts the neural network, reduces the complexity of problem positioning.

[0127] Please refer to Figure 8 , Figure 8is a framework schematic diagram of an embodiment of the processor 80 of the present application. The processor 80 comprises an operation scheduler 81, and an on-chip memory 82, an on-chip adder 83, an on-chip multiplier 84, an on-chip shifter 85 and an on-chip comparator 86 connected to the operation scheduler 81 respectively, the operation scheduler 81 being configured to execute program instructions to implement the steps in any of the embodiments of the hardware invoking method described above, or implement the steps in any of the embodiments of the speech recognition method described above.

[0128] Specifically, the operation scheduler 81 is configured to control itself and the on-chip memory 82, the on-chip adder 83, the on-chip multiplier 84, the on-chip shifter 85 and the on-chip comparator 86 to implement the steps in any of the embodiments of the hardware invoking method described above, or implement the steps in any of the embodiments of the speech recognition method described above. The operation scheduler 81 can be an integrated circuit chip with signal processing capability. The operation scheduler 81 can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. In addition, the operation scheduler 81 can be implemented by integrated circuit chips together.

[0129] The above scheme, by pre-acquiring the mapping relationship between the values in the preset value range and the sequence positions in the reciprocal square root table and the quantized reciprocal square root table. On this basis, the processor 80 only needs to combine the on-chip memory 82, the on-chip adder 83, the on-chip multiplier 84, the on-chip shifter 85 and the on-chip comparator 86 to do fixed-point calculation when performing layer normalization, without doing floating-point calculation, so as to improve the operation efficiency of the algorithm scheme, and the applicability on various processors, and reduce the complexity of problem positioning when the algorithm scheme adopts a neural network.

[0130] Please refer to Figure 9 , Figure 9 is a framework schematic diagram of an embodiment of the computer readable storage medium 90 of the present application. The computer readable storage medium 90 stores program instructions 91 capable of being executed by a processor, the program instructions 91 being configured to implement the steps in any of the embodiments of the hardware invoking method described above, or implement the steps in any of the embodiments of the speech recognition method described above.

[0131] The above scheme, since the computer readable storage medium 90 can realize the steps in the above hardware calling method embodiment, or the steps in the above voice recognition method embodiment, by pre-acquiring the mapping relationship between the values in the preset value range and the sequence positions in the reciprocal square root table and the quantized reciprocal square root table, when performing layer normalization, only fixed-point calculation is needed in combination with the on-chip memory, on-chip adder, on-chip multiplier, on-chip shifter and on-chip comparator, without floating-point calculation, so that the operation efficiency of the algorithm scheme can be improved, the applicability on various processors can be improved, and when the algorithm scheme adopts a neural network, the complexity of problem positioning can be reduced.

[0132] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, details are not repeated here.

[0133] The above description of various embodiments tends to emphasize the differences between various embodiments, and the same or similar parts can be mutually referred to. For brevity, details are not repeated here.

[0134] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the above-described device implementation is only schematic, for example, the division of the module or unit is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual ones can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0135] The unit described as a separate component can be or can not be physically separated, and the component shown as a unit can be or can not be a physical unit, that is, it can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

[0136] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0137] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product in essence or in the form of a software product that contributes to the prior art or the whole or part of the technical solutions. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a processor, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0138] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been explicitly informed of the personal information processing rules before processing the personal information and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as a camera, an explicit and prominent mark is set to inform that the personal information collection range has been entered and the personal information will be collected. If the person voluntarily enters the collection range, it is regarded as agreeing to collect the personal information. Or, on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the person to upload the personal information. The personal information processing rules can include the personal information processor, the processing purpose of personal information, the processing method, and the type of processed personal information, etc.

Claims

1. A speech recognition method, characterized in that, include: Obtain the first quantization parameter set by the user for the square root reciprocal operation within a preset numerical range, and obtain the square root reciprocal table quantized by the first quantization parameter and the mapping relationship between the numerical values ​​within the preset numerical range and the ordinal position in the square root reciprocal table; An algorithm scheme that inputs the acoustic features of audio frames in the speech to be recognized into speech recognition; In response to the fact that the target operator to be executed in the algorithm scheme is a layer normalization operator, the integer output values ​​of each output node in the previous operator of the target operator and the second quantization parameter of the factor in the layer normalization operator are retrieved from the on-chip memory. The on-chip adder, on-chip multiplier, on-chip comparator, on-chip shifter, and the mapping relationship and square root reciprocal table are called to process each of the integer output values ​​and each of the quantization parameters to obtain the layer normalized value of the integer output value; The layer normalized value of each of the integer output values ​​is fed into the next operator of the target operator until the algorithm scheme is completed, and the recognized text of the speech to be recognized is obtained.

2. The method according to claim 1, characterized in that, The process of calling the on-chip adder, on-chip multiplier, on-chip comparator, on-chip shifter, and the mapping relationship and square root reciprocal table to process each of the integer output values ​​and each of the quantization parameters to obtain the layer normalized value of the integer output value includes: The on-chip adder and on-chip multiplier are invoked to obtain the first difference between the product of the integer output value and the total number of nodes and the sum of the integer output values, and the second difference between the total number of nodes, the sum of the squares of the integer output values, and the square of the sum. The on-chip comparator is invoked to split the second difference into the first value within the preset value range, which is then quantized by the third quantization parameter. The mapping relationship is then invoked to look up the quantized value of the reciprocal square root of the first value in the square root reciprocal table, which is used as the second value. The on-chip multiplier and on-chip shifter are invoked to process the first difference, the second value, and each quantization parameter to obtain the layer normalized value of the integer output value.

3. The method according to claim 2, characterized in that, The step of calling the mapping relationship to look up the quantized value of the reciprocal of the square root of the first value in the square root reciprocal table as the second value includes: Use the mapping relationship to obtain the target sequence corresponding to the first value; Find the second value in the reciprocal of the square root table that is located at the target position.

4. The method according to claim 2, characterized in that, The step of calling the on-chip adder and on-chip multiplier to obtain the first difference between the product of the integer output value and the total number of nodes, and the sum of the integer output values, and the second difference between the total number of nodes, the sum of the squares of the integer output values, and the square of the sum, includes: The on-chip adder is invoked to obtain the sum of the integer output values, and the on-chip adder and on-chip multiplier are invoked to obtain the sum of the squares of the integer output values. The on-chip multiplier and on-chip adder are invoked to obtain the first difference between the product of the integer output value and the total number of nodes and the accumulated sum, and the second difference between the product of the total number of nodes and the squared accumulated sum and the square of the accumulated sum.

5. The method according to claim 2, characterized in that, The process of calling the on-chip multiplier and on-chip shifter to process the first difference, the second value, and various quantization parameters to obtain the layer-normalized value of the integer output value includes: The on-chip multiplier is invoked to obtain the first product of the first difference and the second value, and the on-chip shifter is invoked to obtain the third value of the first product, which is left-shifted by the quantization parameter of the integer output value and right-shifted by the first quantization parameter and the third quantization parameter. The on-chip multiplier is invoked to obtain the second product of the third value and the scaling factor, and the on-chip shifter is invoked to obtain the fourth value of the second product shifted to the left according to the second quantization parameter of the scaling factor, and the on-chip shifter is invoked to obtain the fifth value of the translation factor shifted to the left according to the quantization parameter of the integer output value and the second quantization parameter of the scaling factor; The on-chip adder is invoked to obtain the first sum of the fourth and fifth values; The on-chip shifter is invoked to obtain the first sum value, which is shifted left according to the quantization parameter of the layer normalized value and right according to the second quantization parameter of the scaling factor and the quantization parameter of the integer output value, respectively, to obtain the layer normalized value of the integer output value.

6. The method according to claim 1, characterized in that, The mapping relationship is expressed as the integer product of the difference between any value within the preset numerical range and the lower limit of the preset numerical range and the preset value. The preset value is the ratio of the capacity of the square root reciprocal table to the difference between the upper and lower limits of the preset value range.

7. The method according to claim 1, characterized in that, The step of obtaining the table of square root reciprocals quantized by the first quantization parameter includes: Obtain the third product of the reciprocal of the preset value and the ordinal position, and obtain the second sum of the third product and the lower limit of the preset value range; The reciprocal of the square root of the second sum, after being quantized by the first quantization parameter, is rounded down and used as the value of the reciprocal of the square root table in the specified position.

8. The method according to claim 1, characterized in that, Algorithms for speech recognition include the Transformer architecture.

9. A hardware access method, characterized in that, include: Obtain the first quantization parameter set by the user for the square root reciprocal operation within a preset numerical range, and obtain the square root reciprocal table quantized by the first quantization parameter and the mapping relationship between the numerical values ​​within the preset numerical range and the ordinal position in the square root reciprocal table; In response to the fact that the target operator to be executed in the algorithm scheme is a layer normalization operator, the integer output values ​​of each output node in the previous operator of the target operator and the second quantization parameter of the factor in the layer normalization operator are retrieved from the on-chip memory. The on-chip adder, on-chip multiplier, on-chip comparator, on-chip shifter, and the mapping relationship and square root reciprocal table are called to process each of the integer output values ​​and each of the quantization parameters to obtain the layer normalized value of the integer output value.

10. A voice recognition device, characterized in that, include: The acquisition module is used to acquire the first quantization parameter set by the user for the square root reciprocal operation within a preset numerical range, and to acquire the square root reciprocal table quantized by the first quantization parameter and the mapping relationship between the numerical values ​​within the preset numerical range and the ordinal position in the square root reciprocal table; The input module is used to input the acoustic features of the audio frames in the speech to be recognized into the speech recognition algorithm. The retrieval module is used to retrieve, in response to the fact that the target operator to be executed in the algorithm scheme is a layer normalization operator, the integer output values ​​of each output node in the previous operator of the target operator and the second quantization parameter of the factor in the layer normalization operator from the on-chip memory. The calling module is used to call the on-chip adder, on-chip multiplier, on-chip comparator, on-chip shifter, and the mapping relationship and square root reciprocal table to process each of the integer output values ​​and each of the quantization parameters, and obtain the layer normalized value of the integer output value; The output module is used to send the layer normalized value of each of the integer output values ​​into the next operator of the target operator until the algorithm scheme is completed, so as to obtain the recognized text of the speech to be recognized.

11. A hardware access device, characterized in that, include: The acquisition module is used to acquire the first quantization parameter set by the user for the square root reciprocal operation within a preset numerical range, and to acquire the square root reciprocal table quantized by the first quantization parameter and the mapping relationship between the numerical values ​​within the preset numerical range and the ordinal position in the square root reciprocal table; The retrieval module is used to retrieve the integer output values ​​of each output node in the previous operator of the target operator and the second quantization parameter of the factor in the layer normalization operator from the on-chip memory in response to the target operator being a layer normalization operator in the algorithm scheme. The calling module is used to call the on-chip adder, on-chip multiplier, on-chip comparator, on-chip shifter, and the mapping relationship and square root reciprocal table to process each of the integer output values ​​and each of the quantization parameters, so as to obtain the layer normalized value of the integer output value.

12. A processor, characterized in that, It includes an arithmetic scheduler, and on-chip memory, on-chip adder, on-chip multiplier, on-chip shifter and on-chip comparator respectively connected to the arithmetic scheduler. The arithmetic scheduler is used to execute program instructions to implement the speech recognition method according to any one of claims 1 to 8, or to implement the hardware calling method according to claim 9.

13. A computer-readable storage medium, characterized in that, The device stores program instructions that can be executed by a processor, the program instructions being used to implement the speech recognition method according to any one of claims 1 to 8, or to implement the hardware invocation method according to claim 9.

Images