Implementation method, device and equipment of grid generator based on low-precision platform
By performing accuracy conversion and processing of spatial mapping parameters on low-precision platforms, and using the vector instruction set of low-precision platforms for high-precision calculations, the problem of high-precision operator operation accuracy on low-precision platforms is solved and image quality is improved.
Patent Information
- Application Number
- CN202111561920.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-12-20
AI Technical Summary
On low-precision platforms, the accuracy of high-precision operator operations cannot be guaranteed, resulting in low image quality. Especially in spatial transformation networks, the lack of supporting high-precision vector instruction sets such as float32 and int32, limiting low power consumption and high performance Requirements.
By obtaining the original spatial mapping parameters and performing precision type conversion, fixed-point processing and down-position separation when image data input is detected, converting them into target spatial mapping parameters, and using the vector instruction set on the low-precision platform for high-precision calculations, ensuring that The target mapping sits standard accuracy of the grid generator output.
While meeting the requirements of low-precision platforms for low-power consumption and high performance, the accuracy of high-precision operator operations is improved, the image output quality is improved, and efficient computing using high-precision operators on low-precision platforms is achieved.
Smart Images

Figure CN114359640B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer processors, and in particular to an implementation method, device and equipment of a grid generator based on a low-precision platform. Background Art
[0002] With the rapid development of artificial intelligence (AI) technology, the demand for edge computing technology development is becoming more and more urgent, and the development based on embedded neural network processors (neural-network processing units, NPU) or digital signal processing (Digital Signal Process, DSP) is also increasing. In the process of edge computing development, for some nonlinear operators that cannot be implemented by hardware algorithms, vector instruction sets such as NPU vector instruction sets are needed to complete the corresponding technology.
[0003] With the rapid development of NPU and DSP, more high-precision vector instruction sets have emerged. In order to adapt to these high-precision vector instruction sets and ensure the high performance of high-precision operator operations, the power consumption of the platform is correspondingly higher. However, on low-precision platforms, in order to meet the requirements of low power consumption and high performance, the NPU vector instruction set may only support the technology of low-precision operators, or the DSP's appropriate instruction set may also choose a low-precision configuration, such as only supporting float16 and int16 vector instruction sets, but not float32 and int32 vector instruction sets.
[0004] Spatial Transformer Networks (STN) is a learnable module placed in Convolutional Neural Networks (CNN) that can effectively increase spatial invariance. It includes a custom network Localisation Ne, a grid generator, and an image sampler. As an important part of the spatial coordinate mapping transformation of STN, the grid generator requires the calculation of high-precision operators to ensure the accuracy of the final output sampler. At present, the accuracy of high-precision operator operations cannot be guaranteed on low-precision platforms, resulting in low image quality output on low-precision platforms. How to ensure the accuracy of high-precision operator operations while meeting the requirements of low-precision platforms for low power consumption and high performance, improve the accuracy of operations using high-precision operators on low-precision platforms, and ensure image output quality are technical problems that need to be solved at present. Summary of the invention
[0005] The embodiments of the present application provide a method, device and equipment for implementing a grid generator based on a low-precision platform, which can improve the accuracy of operations performed using high-precision operators on a low-precision platform and ensure the quality of image output.
[0006] In a first aspect, the present application provides an implementation method of a grid generator based on a low-precision platform, comprising: when detecting that image data is input into a spatial transformation network, obtaining the original spatial mapping parameters output by a custom network based on the image data; through a first precision type conversion algorithm, the original spatial mapping parameters and the specified parameters are fixed-point processed to obtain fixed-point scalar data and specified fixed-point numbers, and the specified parameters are constants specified in the spatial transformation formula used in the grid generator; through the spatial transformation formula used in the grid generator, the target spatial mapping parameters are calculated according to the fixed-point scalar data and the specified fixed-point numbers; through a second precision type conversion algorithm, the target spatial mapping parameters are separated by bit reduction processing to obtain a first parameter part using a first number of bits and a second parameter part using a second number of bits, the first number of bits being lower than the second number of bits; through a first mapping coordinate calculation formula, the target mapping coordinates output by the grid generator are calculated according to the first parameter part and the second parameter part.
[0007] The embodiment of the present application changes the accuracy of the original spatial mapping parameters output by the custom network based on the image data when detecting that the image data is input to the spatial transformation network, so that the accuracy of the operator input to the grid generator meets its accuracy requirements, and then calculates the target spatial mapping parameters through the spatial transformation formula of the grid generator, and then calculates the target mapping coordinates finally output by the grid generator by changing the accuracy of the high-precision operator, i.e., the target spatial mapping parameters, thereby achieving the accuracy of using high-precision operators for calculations on a low-precision platform, ensuring the accuracy of high-precision operator calculations while meeting the requirements of the low-precision platform for low power consumption and high performance, and ensuring the quality of the image output by the spatial transformation network.
[0008] In a second aspect, the present application provides an implementation device of a grid generator based on a low-precision platform, comprising:
[0009] An original space mapping parameter acquisition unit, used to acquire original space mapping parameters output by a custom network based on image data when detecting that image data is input into a space transformation network;
[0010] A parameter fixed-point processing unit, used to perform fixed-point processing on the original space mapping parameters and the specified parameters through a first precision type conversion algorithm to obtain fixed-point scalar data and specified fixed-point numbers, wherein the specified parameters are constants specified in the space transformation formula used in the grid generator;
[0011] A target space mapping parameter calculation unit, used to calculate the target space mapping parameters according to the fixed-point scalar data and the specified fixed-point number through the space transformation formula used in the grid generator;
[0012] A parameter separation unit is used to perform a bit reduction separation process on the target space mapping parameter by using a second precision type conversion algorithm to obtain a first parameter part using a first number of bits and a second parameter part using a second number of bits, wherein the first number of bits is lower than the second number of bits;
[0013] The target mapping coordinate calculation unit is used to calculate the target mapping coordinates output by the grid generator according to the first parameter part and the second parameter part through the first mapping coordinate calculation formula.
[0014] In the third aspect, the present application provides an implementation device of a grid generator based on a low-precision platform, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method of the first aspect or any optional method of the first aspect is implemented.
[0015] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of the first aspect or any optional method of the first aspect.
[0016] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an implementation device of a mesh generator based on a low-precision platform, the implementation device of the mesh generator based on a low-precision platform enables the implementation device of the mesh generator based on a low-precision platform to perform the steps of the implementation method of the mesh generator based on a low-precision platform of the first aspect mentioned above.
[0017] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0019] Figure 1 It is a flowchart of a method for implementing a grid generator based on a low-precision platform provided in an embodiment of the present application;
[0020] Figure 2It is a schematic diagram of a method flow for converting a low-precision operator into a high-precision operator provided in an embodiment of the present application;
[0021] Figure 3 It is a flowchart of a method for calculating target mapping coordinates provided in an embodiment of the present application;
[0022] Figure 4 It is a structural schematic diagram of an implementation device of a grid generator based on a low-precision platform provided in an embodiment of the present application;
[0023] Figure 5 It is a structural schematic diagram of an implementation device of a grid generator based on a low-precision platform provided in an embodiment of the present application. DETAILED DESCRIPTION
[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0025] It should be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.
[0026] It should also be understood that references to "one embodiment" or "some embodiments" etc. described in the specification of the present application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in the specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0027] Before explaining the implementation method of the grid generator based on the low-precision platform provided in the embodiment of the present application, the relevant concepts involving the space transformation network, grid generator and space transformation formula in the embodiment of the present application are first exemplified.
[0028] The spatial transformation network STN includes a custom network Localisation Net, a grid generator Gridgenerator and a sampler Sampler. The input of the spatial transformation network STN is U and the output is V. After U is input to Localisation Net, Localisation Net outputs the original spatial mapping parameter θ used to map the coordinate relationship between the input U and the output V. i , the Grid generator maps the parameters θ according to the original space i And each coordinate point in the output V, the transformation calculation obtains the mapping coordinates corresponding to each coordinate point in the output V, so that the Sampler can complete the final pixel sampling according to the calculated mapping coordinates and the coordinate points in the input U.
[0029] It should be noted that in order to be able to map the parameters θ according to the original space i And each coordinate point in the output V, transform and calculate the mapping coordinates corresponding to each coordinate point in the output V, and the original space mapping parameter θ in Gridgenerator needs to be transformed through the space transformation formula i Transform to obtain the target space mapping parameter θ i ′, where the spatial transformation formula is formula (1), which is as follows:
[0030] θ0′=θ0*(iW-1) / (oW-1)
[0031] θ1′=θ1*(iW-1) / (oH-1)
[0032] θ2′=(θ2+1-θ0-θ1)*(iW-1) / 2
[0033] θ3′=θ3*(iH-1) / (oW-1)
[0034] θ4′=θ4*(iH-1) / (oH-1)
[0035] θ5′=(θ5+1-θ3-θ4)*(iH-1) / 2 (1)
[0036] Among them, θ i are the N original spatial mapping parameters output by the custom network Localisation Net, i∈(0,N-1); θ i ′ is the mapping parameter θ of the original space in the grid generator i The transformation is performed to obtain N target space mapping parameters; iW is the width of the input image, iH is the height of the input image, oW is the width of the output image, and oH is the height of the output image.
[0037] It should be noted that the original spatial mapping parameters output by the custom network Localisation Net are generally 6. In practical applications, the calculations in the spatial transformation formula require float32 precision, while low-precision platforms may only support float16 precision. In this way, it is difficult to complete float32 precision calculations on low-precision platforms. Therefore, it is necessary to improve the spatial transformation formula of the grid generator on the low-precision platform, and fix the N original spatial mapping parameters input to the Grid generator, so that the Grid generator can use the high precision of the scalar indicator int32 to achieve high-precision calculations on the low-precision platform, and improve the accuracy of using high-precision operators on the low-precision platform.
[0038] It should be noted that low-precision platform is the abbreviation of platform with low-precision configuration. Compared with high-precision platform, low-precision platform is a platform that only supports low-precision operators such as float16 and int16 vector instruction sets to complete corresponding operations, and does not support high-precision operators such as float32 and int32 vector instruction sets to complete corresponding operations.
[0039] It should also be noted that low-precision operators are operators used on low-precision platforms or supported by low-precision platforms, and high-precision operators are operators used on high-precision platforms or supported by high-precision platforms, that is, low-precision operators and high-precision operators are relative. When the accuracy of an operator is lower than that of another operator, the operator is a low-precision operator, and correspondingly, the other operator is a high-precision operator. To realize the operation of high-precision operators on the low-precision platform, it is necessary to ensure the accuracy of the high-precision operator operation.
[0040] See also Figure 1 , Figure 1 This is a flow chart of a method for implementing a grid generator based on a low-precision platform provided in an embodiment of the present application, which is described in detail as follows:
[0041] Step S101, when it is detected that image data is input into a spatial transformation network, original spatial mapping parameters output by the custom network based on the image data are obtained.
[0042] In an embodiment of the present application, when a user uses a spatial transformation network to process an image on a low-precision platform, the low-precision platform inputs the image data to be processed into the spatial transformation network. After the LocalisationNet in the spatial transformation network processes the image data, the corresponding original spatial mapping parameters will be output. The spatial transformation formula in the Grid generator needs to use high-precision operators such as float32 and int32 vector instruction sets to complete the corresponding calculations. In this way, when using the spatial transformation formula for the corresponding calculation, the original spatial mapping parameters input to the Grid generator need to be converted into target spatial mapping parameters that can support high-precision operators such as float32 and int32 operations. On a low-precision platform that only supports float16 and int16 vector instruction sets, if the high-precision requirements of the spatial transformation formula are to be met, the parameters in the spatial transformation formula need to be processed so that it can use high-precision operators to complete the formula calculation. Therefore, when the low-precision platform detects that the image data is input into the spatial transformation network, it obtains the original spatial mapping parameters output by Localisation Net based on the above image data, so as to facilitate the next step of precision conversion processing.
[0043] Step S102, through the first precision type conversion algorithm, the original space mapping parameters and the specified parameters are fixed-point processed to obtain fixed-point scalar data and specified fixed-point numbers, and the specified parameters are constants specified in the space transformation formula used in the grid generator.
[0044] In the embodiment of the present application, the original spatial mapping parameters used in the spatial transformation formula in the spatial transformation network are the N float16 type original spatial mapping parameters θ output by Localisation Net. i , accordingly, the grid generator obtains these N float16 type original space mapping parameters θ i Finally, if you need to use a high-precision operator to complete the calculation of the space transformation formula, you need to map the N float16 type original space parameters θ i Perform corresponding processing to make the original space mapping parameter θ i Convert the target space mapping parameters to high precision type.
[0045] Specifically, the parameters used in the spatial transformation formula include but are not limited to the original spatial mapping parameter θ i , and specify the parameter K, which is specifically the original space mapping parameter θ iParameters to be added or subtracted, for example, 1 added to θ2 in θ2′=(θ2+1-θ0-θ1)*(iW-1) / 2 in formula (1); for another example, 1 added to θ5 in θ5′=(θ5+1-θ3-θ4)*(iH-1) / 2 in formula (1); other parameters such as iW, iH, oW, oH and 1 added or subtracted from these parameters do not need to be fixed-point processed.
[0046] As the original space mapping parameter θ i The specified parameter K added or subtracted, and the original space mapping parameter θ i There is a great correlation. If the specified parameter K is not fixed, the original space mapping parameter θ i After conversion into high-precision fixed-point scalar data, the target space mapping parameter θ finally calculated by the space transformation formula will be i The accuracy of ′ is greatly reduced, and the accuracy of the operation using high-precision operators on low-precision platforms cannot be guaranteed. Therefore, when the specified parameter K is involved in the spatial transformation formula, in order to ensure the accuracy of high-precision calculation on a low-precision platform, the specified parameter K needs to be calibrated to a fixed-point width F of 15 bits or 16 bits. After the specified parameter is fixed-point processed, the specified parameter is updated to K*2 F For example, the specified parameter K is calibrated to a fixed-point value with a bit width F of 15 bits to obtain a specified fixed-point number. For example, when the specified parameter K is 1, the specified fixed-point number is 2. 15 To avoid the subsequent due to the specified parameter K and the original space mapping parameter θ i Calculation errors due to the relationship between .
[0047] Here, the first precision type conversion algorithm refers to an algorithm for converting a lower precision operator into a relatively higher precision operator.
[0048] See also Figure 2 , Figure 2 This is a flow chart of a method for converting a low-precision operator into a high-precision operator provided in an embodiment of the present application, which is described in detail as follows:
[0049] Step S201, separate the decimal and integer of the original space mapping parameter, and perform fixed-point processing on the separated decimal part to obtain a third parameter part and a fourth parameter part, the third parameter part is the integer part of the original space mapping parameter, and the fourth parameter part is a fixed-point number obtained after the decimal part of the original space mapping parameter is fixed-point processed.
[0050] In the embodiment of the present application, the original space mapping parameter θ iThe precision type is the first floating point precision, specifically the maximum floating point precision supported by the low-precision platform, such as float16. i To convert to a high-precision type parameter, the original space mapping parameter θ is required. i Perform preprocessing, such as separating the original spatial mapping parameters θ i The integer part and the decimal part, that is, the decimal and integer parts of the original space mapping parameters are separated, and then the separated decimal part is fixed-point processed according to the calibration bit width F, for example, the decimal part is fixed-point processed with a calibration bit width F of 15 bits to obtain the fixed-point number of int16, that is, the fourth parameter part, to realize the conversion of float type data into int type data.
[0051] It should be noted that the value of the calibration bit width F in the embodiment of the present application is less than or equal to the value corresponding to the maximum bit width supported by the low-precision platform, which is determined specifically according to the support and precision of the low-precision platform. For example, if the platform only supports int16 operations, the calibration bit width F of the platform can be 15 bits; for another example, if the platform supports unit16 operations, the calibration bit width F can be 16 bits; for another example, if the platform supports int16 and int32 operations, the calibration bit width F of the platform can be 16 bits, 18 bits, 31 bits, etc. The selection of the calibration bit width is determined by the support and precision requirements of the low-precision platform, and it is necessary to comprehensively consider the operation precision, operation rate, power consumption, etc., and achieve the maximum operation precision with the lowest power consumption under the premise of ensuring the operation precision and operation rate.
[0052] The embodiment of the present application is described below based on a platform that supports int16 precision operations. On this low-precision platform, the calibration bit width F is 15 bits.
[0053] It should also be noted that each float16 raw space mapping parameter θ i , can be regarded as consisting of an integer part and a decimal part. Each original space mapping parameter θ can be mapped by the existing method of separating the integer and decimal parts of floating-point numbers. i Decomposed into parameters consisting of an integer part and a decimal part, that is, the original space mapping parameter θ i It consists of a float16 integer part and a float16 decimal part.
[0054] Then the original space mapping parameter θ obtained by decomposition i The integer part in is converted into the integer part corresponding to the first integer precision to obtain the third parameter part; and the original space mapping parameter θ obtained by decomposition is calibrated by the bit width. iThe decimal part in is processed into a fixed point, and the fixed-point number obtained after the fixed-point processing is converted into a fixed-point number corresponding to the first integer precision to obtain the fourth parameter part.
[0055] Specifically, the original space mapping parameter θ obtained by decomposition according to the calibration bit width i The decimal part in is processed as fixed point.
[0056] In some embodiments of the present application, after the integer part and the decimal part of the original spatial mapping parameter are separated, the calibration bit width F of the low-precision platform is obtained, and then the calibration number used for the fixed-point processing of the separated decimal part is determined according to the calibration bit width F. Then, the separated decimal part is fixed-point processed according to the calibration number.
[0057] Here, the calibration number is 2 F , the decomposed original space mapping parameter θ i The decimal part in the equation is the same as the confirmed calibration number 2 F After multiplication, the integer part is the original space mapping parameter θ obtained by decomposition. i The decimal part in is converted to a fixed-point number, and the precision type of the fixed-point number is converted to the same first integer precision as the integer part, such as int16 type.
[0058] In some embodiments of the present application, according to the calibration bit width F, the original space mapping parameter θ obtained by separation is accumulating registers using vector instructions. i The decimal part in is fixed-point processed, and the original space mapping parameter θ obtained by separation is i The decimal part in is converted into a fixed-point number corresponding to the first integer precision to obtain the fourth parameter part.
[0059] The original space mapping parameter θ obtained by separation i After the decimal part in is fixed-point processed, the obtained fixed-point number is an integer. At this time, the original space mapping parameter θ of float16 type can be i Converts to a parameter consisting of the third and fourth parameter parts of int16 type.
[0060] Step S202, integrating the third parameter part and the fourth parameter part to obtain fixed-point scalar data.
[0061] In the embodiment of the present application, the precision type of the third parameter part and the fourth parameter part is the first integer precision, and can be operated through vector instructions on a low-precision platform to obtain fixed-point scalar data.
[0062] Specifically, since the spatial transformation formula requires the support of high-precision operators such as float32 to obtain accurate calculation results, the third parameter part and the fourth parameter part need to be integrated so that the integrated data can utilize the high precision of scalar instructions supported by low-precision platforms such as int32 to meet the spatial transformation formula's requirements for high-precision operators.
[0063] In some embodiments of the present application, the third parameter part and the fourth parameter part are stored in a vector register; then, through a vector instruction, the third parameter part and the fourth parameter part stored in the vector register are moved to a scalar register for integration according to a first predetermined rule to obtain fixed-point scalar data.
[0064] It should be noted that the third parameter part and the fourth parameter part corresponding to each parameter in the original space mapping parameter are integrated to obtain a fixed-point scalar data, that is, when the original space mapping parameter is 6 parameters, 6 corresponding fixed-point scalar data are obtained.
[0065] In some embodiments of the present application, the third parameter part and the fourth parameter part are saved in a vector register, and then the third parameter part and the fourth parameter part in the vector register are moved to a scalar register according to a first predetermined rule to implement an integration operation on the third parameter part and the fourth parameter part, so as to obtain fixed-point scalar data with a precision type of the second integer precision, such as int32.
[0066] Specifically, when the third parameter part and the fourth parameter part in the vector register are moved to the scalar register according to the first predetermined rule, the sign bit needs to be extended to ensure that the precision type of the integrated data is converted to the second integer precision. After obtaining the fixed-point scalar data, the calculation of the space transformation formula can be completed by high-precision addition, subtraction, multiplication and division such as int32 to obtain a high-precision calculation result.
[0067] In some other embodiments of the present application, the third parameter part and the fourth parameter part stored in the vector register are moved to the scalar register according to a first predetermined rule through a vector instruction.
[0068] Specifically, when the third parameter part and the fourth parameter part stored in the vector register are moved to the scalar register according to the first predetermined rule through the vector instruction, the third parameter part is shifted to the left by the corresponding number of bits in the scalar register according to the calibration bit width. For example, when the calibration bit width F is 15 bits, the third parameter part is shifted to the left by 15 bits in the scalar register, and the shifted third parameter part and the fourth parameter part moved to the scalar register are used as fixed-point scalar data. The obtained fixed-point scalar data is a fixed-point number, and high-precision calculations can be performed through scalar instructions such as int32.
[0069] It should be noted that, according to the calibration bit width F, the third parameter part is shifted to the left by the corresponding number of bits in the scalar register. While ensuring that the first integer precision can be converted into the second integer precision, it is convenient to convert the second integer precision into the first integer precision again through reverse operations when necessary, and separate the fixed-point scalar data into the integer part and the decimal part of the first integer precision, so that the mapping coordinates can be calculated through the vector instruction int16 on the low-precision platform.
[0070] It should also be noted that the value of the number of bits in the shift corresponding bit number is the same as the value of the calibration bit width. Each original space mapping parameter corresponds to a fixed-point scalar data. When calculating the fixed-point scalar data, it is necessary to perform the relevant operations of step S101 and step S102 on each original space mapping parameter until the fixed-point scalar data of all original space mapping parameters are obtained.
[0071] Step S103, using the space transformation formula used in the grid generator, calculate the target space mapping parameter θ according to the fixed-point scalar data and the specified fixed-point number i ′.
[0072] In the embodiment of the present application, the target space mapping parameter θ i The precision type of ′ is the second integer precision. Each original space mapping parameter θ i Corresponding to a fixed-point scalar data, each specified parameter corresponds to a specified fixed-point number, and the original space mapping parameter θ is obtained i After the corresponding fixed-point scalar data and the specified fixed-point number corresponding to the specified parameter K are obtained, the fixed-point scalar data and the specified fixed-point number are placed in the corresponding positions of the spatial transformation formula for calculation to obtain the target spatial mapping parameter θ i ′.
[0073] On a low-precision platform, since the transformation formula needs to use high-precision operators such as float32 and int32 to complete the corresponding calculations, after the relevant operations of steps S101 to S103, high-precision int32 scalar data is obtained. The int32 scalar data is added, subtracted, multiplied and divided by scalar instructions, and the target space mapping parameter θ can be calculated. i ′, which is the target space mapping parameter θ required to be solved in the space transformation formula i ′, thus achieving the purpose of using high-precision operators to perform related calculations on a low-precision platform.
[0074] Step S104, performing bit separation processing on the target space mapping parameter through a second precision type conversion algorithm to obtain a first parameter part using a first bit number and a second parameter part using a second bit number, wherein the first bit number is lower than the second bit number.
[0075] In the embodiment of the present application, since the calculated target space mapping parameter θ i ′ is a high-precision operator, and only the spatial transformation formula in the Grid generator of the spatial transformation network needs to use the high-precision operator for calculation. Under the premise that the low-precision platform only supports low-precision operator calculations, using the high-precision operator for subsequent calculations may cause errors, and the accuracy of using high-precision operators on low-precision platforms cannot be guaranteed. Therefore, when calculating the target space mapping parameter θ i ′, the high-precision operator needs to be converted into an operator supported by the instruction set of the low-precision platform, that is, the target space mapping parameter θ i 'Perform a bit reduction separation process to obtain a first parameter part using the first number of bits and a second parameter part using the second number of bits.
[0076] It should be noted that the first parameter part corresponds to the decimal part of the target space mapping parameter, the second parameter part corresponds to the integer part of the target space mapping parameter, and the precision type of the first parameter part and the second parameter part is the first integer precision.
[0077] Since the precision type of the fixed-point scalar data is the second integer precision, the target space mapping parameter θ calculated by the space transformation formula is i The precision type of ′ is also the second integer precision. On the low-precision platform, the vector instruction does not support the high-precision of the second integer precision, such as int32 operations. The target space mapping parameter θ is calculated by the space transformation formula. i ′, the target space mapping parameter θ needs to be i The precision type of ′ is the second integer precision, such as int32, which is converted to the first integer precision supported by the low-precision platform, such as int16, to facilitate the calculation of the mapping coordinates finally output by the grid generator, so as to ensure the accuracy of high-precision operator operations on the low-precision platform.
[0078] The target space mapping parameter θ is calculated by the space transformation formula i ′After that, according to the calibration bit width F, the target space mapping parameter θ is mapped according to the second predetermined rule. i ' Perform bit separation processing to obtain the first bit number b m The first parameter part θD i and use the second bit number b nThe second parameter part θI i , where b m The low bit number b0~b F-1 , b n The number of high bits b F ~b F-1+M , where M is determined by the maximum bit width supported by the vector register and the maximum bit width supported by the accumulator operation. For example, the calibration bit width is 15 bits, the maximum bit width supported by the vector register is 16 bits, and the maximum bit width supported by the accumulator operation is 32 bits, that is, the accumulator supports operations with a maximum bit width of 32 bits, then M is 16, because 32-15>16, that is, the difference between the maximum bit width supported by the accumulator operation minus the calibration bit width F is compared with the maximum bit width of the vector register. When the maximum bit width supported by the accumulator operation minus the calibration bit width F is The difference is greater than or equal to the maximum bit width of the vector register, then the value of M corresponds to the value corresponding to the maximum bit width of the vector register; when the difference between the maximum bit width supported by the accumulator register operation and the calibration bit width F is less than the maximum bit width of the vector register, then the value of M corresponds to the difference between the maximum bit width supported by the accumulator register operation and the calibration bit width F. For example, the calibration bit width is 18 bits, the maximum bit width supported by the vector register is 32 bits, and the maximum bit width supported by the accumulator register operation is 48 bits. Then the value of M corresponds to 48-18=30, because 48-18<32.
[0079] In some embodiments of the present application, the first bit number b is obtained according to the number of bits corresponding to the calibration bit width F through a vector instruction. m The corresponding first parameter part obtains the first parameter part θD using the first bit number i For example, when the calibration bit width F is 15 bits, the lower 15 bits are obtained according to 15 bits through the AND operation in the vector instruction, that is, the bit number is b0~b 14 The first parameter part corresponding to the first bit number b0~b 14 The first parameter part, that is, the fractional part of the target space mapping parameter θD i .
[0080] In some other embodiments of the present application, the target space mapping parameter θ stored in the scalar register is converted into the target space mapping parameter θ according to the calibration bit width through the vector instruction. i 'Move the corresponding number of bits to the right, and then use the vector instruction to obtain the second bit number b from the target mapping parameter after shifting the corresponding number of bits to the right in the scalar register according to the number of bits corresponding to the calibration bit width F. n The corresponding second parameter part is obtained by using the second bit number of the second parameter part θI iFor example, when the calibration bit width F is 15 bits, the target space mapping parameter θ is set in the scalar register. i 'Move right 15 bits, and through the AND operation in the vector instruction, get the lower 15 bits after the shift according to 15 bits, that is, the bit number is b 15 ~b 30 The corresponding second parameter part obtains the second parameter part using the second bit number, that is, the integer part θI of the target space mapping parameter i .
[0081] Step S105, calculating the target mapping coordinates output by the grid generator according to the first parameter part and the second parameter part through the first mapping coordinate calculation formula.
[0082] In the embodiment of the present application, the instruction set supported by the processor on the low-precision platform includes scalar instructions in addition to vector instructions. Vector instructions can process N mac The scalar instruction can only process one number, and generally supports int32 operations. The precision of the accumulator register is generally higher, and integer operations can reach 32 bits or more.
[0083] The calculation of the target mapping coordinates is based on the high-precision operator, using the instructions supported by the low-precision platform to complete the high-precision operator calculation. In order to be able to use the instructions supported by the low-precision platform to complete the high-precision operator calculation and ensure the accuracy of the high-precision operator calculation on the low-precision platform, the high-precision of the accumulator register can be used to ensure the calculation operation of the high-precision operator.
[0084] Specifically, the embodiment of the present application utilizes an accumulator register to calculate the target mapping coordinates output by the grid generator according to the first parameter part and the second parameter part through the first mapping coordinate calculation formula. It can complete the calculation of high-precision operators on the low-precision platform using instructions supported by the low-precision platform, thereby ensuring the accuracy of operations performed using high-precision operators on the low-precision platform.
[0085] In some embodiments of the present application, when mapping the target space parameter θ i ' performs integer and decimal separation processing to obtain the first parameter part θD using the first bit number i and the second parameter part θI using the second bit number i , you can use low-precision vector instructions supported by low-precision platforms, such as int16 vector instructions, and achieve the accuracy of high-precision operations using low-precision vector instructions on low-precision platforms by accumulating registers.
[0086] See also Figure 3 , Figure 3This is a flow chart of a method for calculating target mapping coordinates provided in an embodiment of the present application, which is described in detail as follows:
[0087] Step S301: According to the first parameter part and the second parameter part, corresponding parameters in the first mapping coordinate calculation formula are deformed to obtain a second mapping coordinate calculation formula.
[0088] In the embodiment of the present application, the mapping coordinate calculation formula used in the grid generator is the first mapping coordinate calculation formula, and the first mapping coordinate calculation formula is shown in Formula 2:
[0089] x′=θ′0*x+θ1′*y+θ′2
[0090] y′=θ′3*x+θ′4*y+θ′5 (2)
[0091] Among them, θ i ′ is the target space mapping parameter calculated by the space transformation formula, x and y are the coordinate points in the output V of the grid generator, and x′ and y′ are the coordinate points in the input U, that is, the target mapping coordinates that need to be calculated.
[0092] Since the calculated target space mapping parameter θ i The precision type of ′ is int32, and the vector instructions on the low-precision platform do not support int32 operations. Therefore, when using the target space mapping parameter θ i ′, a bit reduction separation process is performed on it to obtain the first parameter part θD using the first bit number i and the second parameter part θI using the second bit number i After obtaining the mapping coordinate calculation formula used in the grid generator, namely the first mapping coordinate calculation formula, the first mapping coordinate calculation formula can be transformed into a second mapping coordinate calculation formula, so that the second mapping coordinate calculation formula can be used to meet the requirement of obtaining high-precision results on a low-precision platform by using a high-precision accumulation register. The transformed second mapping coordinate calculation formula is shown in formula (3):
[0093] x′=(θI0*2 F +θD0)*x+(θI1*2 F +θD1)*y+(θI2*2 F +θD2)
[0094] y′=(θI3*2 F +θD3)*x+(θI4*2 F +θD4)*y+(θI5*2 F +θD5) (3)
[0095] Where, θDi The first parameter part using the first bit number; θI i The second parameter part using the second bit number; 2 F is the calibration number, F is the calibration bit width; x and y are the coordinate points in the output V of the grid generator, and x′ and y′ are the coordinate points in the input U, that is, the target mapping coordinates that need to be calculated.
[0096] It should be noted that in the process of transforming the first mapping coordinate calculation formula into the second mapping coordinate calculation formula, since the first parameter part θD is required i and the second parameter part θI i Instead of the corresponding target space mapping parameter θ in formula (2) i ′, and the first parameter part θD i For fixed-point data, the second parameter part θI is used i The calibration number must be multiplied by the calibration number, which is determined by the calibration bit width F, which is 2 to the Fth power. It can be achieved by shifting the calibration bit width F digits to the left by the accumulator register. For example, when the calibration bit width is 15 bits, the calibration number is 2 15 ; You can also shift the accumulator register 15 bits to the left to achieve the calibration number 2 15 .
[0097] Step S302, initializing the second mapping coordinate calculation formula to obtain an initialization coordinate calculation formula.
[0098] In the embodiment of the present application, the initialization coordinate calculation formula is shown in formula (4):
[0099] vacc x =θI0*2 F *vseq+θD0*vseq+θI2*2 F +θD2
[0100] vacc y =θI3*2 F *vseq+θD3*vseq+θI5*2 F +θD5 (4)
[0101] Among them, vacc is the accumulator register, vacc x and vacc y The target mapping coordinates stored in the accumulator register; θD i The third parameter part using the first bit number; θI i is the fourth parameter part using the second bit number; F is the calibration bit width, 2 F is the calibration number, vseq is a vector length N macsequence, vseq∈[0,N mac -1].
[0102] Step S303, calculating and obtaining the first mapping coordinates according to the initialization coordinate calculation formula.
[0103] Step S304: Update the first mapping coordinates using the x-axis coordinate update formula and the y-axis coordinate update formula.
[0104] In the embodiment of the present application, the target mapping coordinates (vacc) are calculated by using the second mapping coordinates through the high-precision calculation of the accumulator register. x ,vacc y ), vector processing is used on the x-axis, specifically according to a vector length N mac The y-axis is processed row by row.
[0105] Before calculating the target mapping coordinates output by the grid generator through formula (3), the mapping coordinates of the x / y axis need to be initialized. The initial first mapping coordinates of the x / y axis are calculated through the initialization coordinate calculation formula, and the y axis coordinates are updated row by row through the y axis coordinate update formula. The x axis coordinate update formula is used according to a vector length N. mac The x-axis coordinates are updated by a sequence to obtain a plurality of first mapping coordinates.
[0106] In the embodiment of the present application, after the first mapping coordinates are calculated according to the initialization coordinate calculation formula, since the y-axis coordinates are updated row by row, the first mapping coordinates can be directly updated by the y-axis coordinate update formula, where the y-axis coordinate update formula is shown in formula (5):
[0107] vacc x =vacc x +θI1*2 F +θD1
[0108] vacc y =vacc y +θI4*2 F +θD4 (5)
[0109] In some embodiments of the present application, if the vector instruction supports unsigned int16 operations, the scalar number 2 F It can be used as a constant for multiplication and accumulation, and then an addition and accumulation is added to implement formula (5). If the vector instruction supports signed int16 operations, then the θI in the formula needs to be updated for the y coordinate i *2 F Perform deformation processing and transform it into θI i ′*2 F-1 , where θI i′=θI*2, so the update operation of the y-axis coordinate can be completed through multiplication and accumulation and addition and accumulation.
[0110] It should be noted that since the y-axis coordinate of the target mapping coordinate is updated row by row, the θI in the y-axis coordinate update formula is i *2 F Deformation is θI i ′*2 F-1 It can be used as a constant for multiplication and accumulation, and then an addition and accumulation is added to implement formula (5), thereby realizing the update of the y-axis coordinate of the target mapping coordinate.
[0111] In the embodiment of the present application, a vector length N mac The x-axis coordinate update formula is used to update the x-axis coordinate in the target mapping coordinate, where the x-axis coordinate update formula is shown in formula (6):
[0112] vacc x =vacc x -θI1*2 F *oH-θD1*oH
[0113] vacc y =vacc y -θI4*2 F *oH-θD4*oH
[0114] vacc x =vacc x +θI0*2 F *N mac +θD0*N mac
[0115] vacc y =vacc y +θI3*2 F *N mac +θD3*N mac (6)
[0116] Where oH is the height of the output image, N mac is the length of the vector.
[0117] It should be noted that, since the y-axis coordinate update loop is within the x-axis coordinate update loop, when updating the x-axis coordinate, it is necessary to subtract the accumulation of oH y-axis coordinate update operations and then perform the x-axis coordinate update operation, as shown in formula (6).
[0118] In some embodiments of the present application, if the vector instruction supports unsigned int16 operations, the scalar number 2 FIt can be used as a constant for multiplication and accumulation, and then an addition and accumulation is added to implement formula (6). If the vector instruction supports signed int16 operations, then the θI in the formula needs to be updated for the x coordinate i *2 F *oH is deformed to θI i ″*2 F , where θI i ″=θI*oH;and update the θI in the x-coordinate formula i *2 F *N mac Perform deformation processing and transform it into θI i ″′=θI*N mac After the formula transformation is completed, the update operation of the y-axis coordinate can be completed through multiplication and accumulation and addition and accumulation.
[0119] Step S305 , using the vector register and the coordinate and weight calculation formula, the target mapping coordinate output by the grid generator is determined according to the obtained first mapping coordinate.
[0120] In the embodiment of the present application, the first mapping coordinates include the coordinates calculated according to the initialization coordinate calculation formula, and also include the coordinates updated by the x-axis coordinate update formula and the y-axis coordinate update formula. When each first mapping coordinate is obtained, the vector register is used to determine the target mapping coordinate corresponding to the first mapping coordinate according to the coordinate and weight calculation formula, thereby determining the target mapping coordinate output by the final grid generator. The coordinate and weight calculation formula is shown in formula (7):
[0121] vx I =vacc x .(b F ~b F-1+M )
[0122] vx D =vacc x .(b0~b F-1 )
[0123] vy I =vacc y .(b F ~b F-1+M )
[0124] vy D =vacc y .(b0~b F-1 )
[0125] vfx w =float(vx D )*float(2 -F )
[0126] vfy w = float(vy D )*float(2 -F ) (7)
[0127] where v is the label of the vector register; vx I is the integer part of the destination mapping coordinate stored in the vector register; vx D is the fractional part of the target mapping coordinates stored in the vector register; vfx w is the weight of the x coordinate in the target mapping coordinates stored in the vector register; vfy w is the weight of the y coordinate in the target map coordinates stored in the vector register.
[0128] It should be noted that the weight is actually the decimal part of the target mapping coordinate, that is, vfx w and vfy w It is the floating point decimal part of the target mapping coordinates. Whether to use the weight parameter can be determined by the implementation of the sampler. For example, the nearest principle does not use the weight parameter, and the bilinear interpolation uses the weight parameter.
[0129] When implementing formula (7), the accumulator register must support the truncation of a preset number of bits (e.g., the number of bits determined by the calibration bit width F) to the vector register, where the first bit number refers to the bit position b0 to b F-1 The second bit refers to the bit b F ~b F-1+M The cut-off position.
[0130] It should also be noted that in order to ensure the accuracy of high-precision operator operations on low-precision platforms, the vector instruction set of the NPU or DSP on low-precision platforms, such as low-precision platforms that support int16 operations, needs to have the following functions:
[0131] 1) Scalar instructions support int32 operations;
[0132] 2) The bit width of the accumulator register must be 32 bits or above;
[0133] 3) Elements in a vector register can be moved to a scalar register, and the sign bit can be extended; the reverse is also supported, that is, elements in a scalar register can be moved to a vector register.
[0134] 4) The accumulator register must support 15-bit or 16-bit truncation and transfer to the vector register, where 15-bit truncation means the bits are b0 to b 14 The first bit and bit of b15 ~b 30 The second bit of .
[0135] It should also be noted that when the low-precision platform is a platform that supports int32, the corresponding vector instruction set of the NPU or DSP needs to have the following functions:
[0136] 1) Scalar instructions support at least int32 operations;
[0137] 2) The bit width of the accumulator register must be 48 bits or more;
[0138] 3) Elements in a vector register can be moved to a scalar register, and the sign bit can be extended; the reverse is also supported, that is, elements in a scalar register can be moved to a vector register.
[0139] The accumulator register must support arbitrary bit truncation and transfer to the vector register. For example, when the calibration bit width is 18 bits, the first bit is b0~b 17 , the second bit is b 18 ~b 47 .
[0140] In an embodiment of the present application, when it is detected that image data is input into the spatial transformation network, the accuracy of the original spatial mapping parameters output by the custom network based on the image data is changed, so that the accuracy of the operator input into the grid generator meets its accuracy requirements, and then the target spatial mapping parameters are calculated by the spatial transformation formula of the grid generator, and then the target mapping coordinates finally output by the grid generator are calculated by changing the accuracy of the calculated high-precision operator, i.e., the target spatial mapping parameters, thereby achieving the accuracy of using high-precision operators for calculations on a low-precision platform, while meeting the requirements of the low-precision platform for low power consumption and high performance, thereby ensuring the accuracy of the high-precision operator calculations, thereby ensuring the quality of the image output by the spatial transformation network.
[0141] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0142] Based on the implementation method of the grid generator based on the low-precision platform provided in the above embodiment, the embodiment of the present application further provides an embodiment of a device for implementing the above method embodiment.
[0143] See also Figure 4 , Figure 4 Schematic diagram of a low-precision platform-based grid generator implementation device provided in an embodiment of the present application. The units included are used to execute Figure 1For details, please refer to the steps in the corresponding embodiment. Figure 1 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 4 , the implementation device 4 of the grid generator based on the low-precision platform includes:
[0144] The original space mapping parameter acquisition unit 41 is used to acquire the original space mapping parameters output by the custom network based on the image data when it is detected that the image data is input into the space transformation network;
[0145] The parameter fixed-point processing unit 42 is used to perform fixed-point processing on the original space mapping parameters and the specified parameters through the first precision type conversion algorithm to obtain fixed-point scalar data and specified fixed-point numbers, where the specified parameters are constants specified in the space transformation formula used in the grid generator;
[0146] The target space mapping parameter calculation unit 43 is used to calculate the target space mapping parameters according to the fixed-point scalar data and the specified fixed-point number through the space transformation formula used in the grid generator;
[0147] The parameter separation unit 44 is used to perform a bit reduction separation process on the target space mapping parameter by using a second precision type conversion algorithm to obtain a first parameter part using a first bit number and a second parameter part using a second bit number, wherein the first bit number is lower than the second bit number;
[0148] The target mapping coordinate calculation unit 45 is used to calculate the target mapping coordinates output by the grid generator according to the first parameter part and the second parameter part through the first mapping coordinate calculation formula.
[0149] In some embodiments of the present application, the parameter fixed-point processing unit 42 is specifically used for:
[0150] The original space mapping parameter is processed by separating the decimal and the integer, and the decimal part obtained by the separation is processed by fixed-point conversion to obtain a third parameter part and a fourth parameter part, wherein the third parameter part is the integer part of the original space mapping parameter, and the fourth parameter part is a fixed-point number obtained by converting the decimal part of the original space mapping parameter to a fixed-point number;
[0151] The third parameter part and the fourth parameter part are integrated to obtain fixed-point scalar data.
[0152] In some other embodiments of the present application, the parameter fixed-point processing unit 42 is further used for:
[0153] Get the calibration bit width of the low-precision platform;
[0154] According to the calibration bit width, determine the calibration number used for fixed-point processing of the separated decimal part;
[0155] The separated decimal part is fixed-point processed according to the calibration number.
[0156] In some other embodiments of the present application, the parameter fixed-point processing unit 42 is further used for:
[0157] storing the third parameter portion and the fourth parameter portion in a vector register;
[0158] Through the vector instruction, the third parameter part and the fourth parameter part stored in the vector register are moved to the scalar register for integration according to the first predetermined rule to obtain fixed-point scalar data.
[0159] In some other embodiments of the present application, the parameter fixed-point processing unit 42 is further used for:
[0160] According to the calibration bit width, the third parameter part is shifted to the left by the corresponding number of bits in the scalar register;
[0161] The third parameter part and the fourth parameter part after being shifted to the left by the corresponding number of bits are integrated to obtain fixed-point scalar data.
[0162] In some other embodiments of the present application, the parameter separation unit 44 is specifically used for:
[0163] According to the calibration bit width, the target space mapping parameter is subjected to bit reduction separation processing according to a second predetermined rule to obtain a first parameter part using a first number of bits and a second parameter part using a second number of bits.
[0164] In some other embodiments of the present application, the parameter separation unit 44 is further configured to:
[0165] According to the calibration bit width, obtaining a first parameter part of a corresponding number of bits from the target mapping parameter stored in the scalar register to obtain a first parameter part using a first number of bits;
[0166] According to the calibration bit width, the target space mapping parameter stored in the scalar register is shifted right by the corresponding number of bits;
[0167] According to the calibration bit width, a second parameter part of a corresponding number of bits is obtained from the target mapping parameter after being shifted rightward by a corresponding number of bits in the scalar register, so as to obtain a second parameter part using a second number of bits.
[0168] In some embodiments of the present application, the target mapping coordinate calculation unit 45 is specifically used to:
[0169] According to the first parameter part and the second parameter part, corresponding parameters in the first mapping coordinate calculation formula are deformed to obtain a second mapping coordinate calculation formula;
[0170] Initialize the second mapping coordinate calculation formula to obtain the initialization coordinate calculation formula;
[0171] Calculate the first mapping coordinates according to the initialization coordinate calculation formula;
[0172] Update the first mapping coordinates using the x-axis coordinate update formula and the y-axis coordinate update formula;
[0173] The target mapping coordinates output by the grid generator are determined according to the obtained first mapping coordinates by using the vector register and the coordinate and weight calculation formula.
[0174] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0175] Figure 5 Schematic diagram of a device for implementing a grid generator based on a low-precision platform provided in an embodiment of the present application. Figure 5 As shown, the implementation device 5 of the low-precision platform-based mesh generator of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a speech recognition program. When the processor 50 executes the computer program 52, the steps in the above-mentioned various implementation methods of the low-precision platform-based mesh generator are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 4 The functions of units 41-45 are shown.
[0176] Exemplarily, the computer program 52 may be divided into one or more modules / units, one or more modules / units are stored in the memory 51 and executed by the processor 50 to complete the present application. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the implementation device 5 of the grid generator based on the low-precision platform. For example, the computer program 52 may be divided into an original space mapping parameter acquisition unit 41, a parameter fixed-point processing unit 42, a target space mapping parameter calculation unit 43, a parameter separation unit 44, and a target mapping coordinate calculation unit 45. For specific functions of each unit, please refer to Figure 1 The relevant descriptions in the corresponding embodiments are not repeated here.
[0177] The implementation device of the grid generator based on the low-precision platform may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that Figure 5 It is only an example of an implementation device 5 of a grid generator based on a low-precision platform and does not constitute a limitation on the implementation device 5 of a grid generator based on a low-precision platform. It may include more or less components than shown in the figure, or a combination of certain components, or different components. For example, the implementation device of a grid generator based on a low-precision platform may also include input and output devices, network access devices, buses, etc.
[0178] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0179] The memory 51 may be an internal storage unit of the implementation device 5 of the grid generator based on the low-precision platform, such as a hard disk or memory of the implementation device 5 of the grid generator based on the low-precision platform. The memory 51 may also be an external storage device of the implementation device 5 of the grid generator based on the low-precision platform, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the implementation device 5 of the grid generator based on the low-precision platform. Further, the memory 51 may also include both the internal storage unit of the implementation device 5 of the grid generator based on the low-precision platform and an external storage device. The memory 51 is used to store computer programs and other programs and data required by the implementation device of the grid generator based on the low-precision platform. The memory 51 may also be used to temporarily store data that has been output or is to be output.
[0180] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method for implementing the grid generator based on the low-precision platform can be implemented.
[0181] An embodiment of the present application provides a computer program product. When the computer program product runs on an implementation device of a grid generator based on a low-precision platform, the implementation device of the grid generator based on the low-precision platform can implement the above-mentioned implementation method of the grid generator based on the low-precision platform when executed.
[0182] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0183] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0184] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0185] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for implementing a grid generator based on a low-precision platform, It is characterized in that The method comprises: When detecting that image data is input into the spatial transformation network, obtaining original spatial mapping parameters output by the custom network based on the image data; The original space mapping parameters and the specified parameters are fixed-point processed by a first precision type conversion algorithm to obtain fixed-point scalar data and specified fixed-point numbers, wherein the specified parameters are constants specified in the space transformation formula used in the grid generator; Calculating target space mapping parameters according to the fixed-point scalar data and the specified fixed-point number by a space transformation formula used in the grid generator; Performing a bit reduction separation process on the target space mapping parameter by a second precision type conversion algorithm to obtain a first parameter part using a first number of bits and a second parameter part using a second number of bits, wherein the first number of bits is lower than the second number of bits; The target mapping coordinates output by the grid generator are calculated according to the first parameter part and the second parameter part through a first mapping coordinate calculation formula.
2. The method for implementing the grid generator based on the low-precision platform as claimed in claim 1, It is characterized in that The method of performing fixed-point processing on the original space mapping parameter and the specified parameter by using the first precision type conversion algorithm to obtain fixed-point scalar data and a specified fixed-point number includes: The original space mapping parameter is subjected to decimal and integer separation processing, and the decimal part obtained by separation is subjected to fixed-point processing to obtain a third parameter part and a fourth parameter part, wherein the third parameter part is the integer part of the original space mapping parameter, and the fourth parameter part is a fixed-point number obtained by subjecting the decimal part of the original space mapping parameter to fixed-point processing; The third parameter part and the fourth parameter part are integrated to obtain the fixed-point scalar data.
3. The method for implementing the grid generator based on the low-precision platform as claimed in claim 2, It is characterized in that The decimal and integer separation processing is performed on the original space mapping parameters, and the decimal part obtained by separation is fixed-point processed to obtain the third parameter part and the fourth parameter part, including: Obtaining the calibration bit width of the low-precision platform; According to the calibration bit width, determining the calibration number used for fixed-point processing of the separated decimal part; The separated decimal part is subjected to fixed-point processing according to the calibration number.
4. The method for implementing a grid generator based on a low-precision platform as claimed in claim 2 or 3, It is characterized in that The step of integrating the third parameter part and the fourth parameter part to obtain the fixed-point scalar data includes: storing the third parameter portion and the fourth parameter portion in a vector register; Through the vector instruction, the third parameter part and the fourth parameter part stored in the vector register are moved to the scalar register for integration according to a first predetermined rule to obtain the fixed-point scalar data.
5. The method for implementing the grid generator based on the low-precision platform as claimed in claim 4, It is characterized in that The method of moving the third parameter part and the fourth parameter part stored in the vector register to the scalar register for integration according to a first predetermined rule through the vector instruction to obtain the fixed-point scalar data includes: According to the calibration bit width, the third parameter part is shifted to the left by a corresponding number of bits in the scalar register; The third parameter part and the fourth parameter part after being shifted to the left by a corresponding number of bits are integrated to obtain the fixed-point scalar data.
6. The method for implementing a grid generator based on a low-precision platform as claimed in claim 1 or 2, It is characterized in that The step of performing a bit reduction separation process on the target space mapping parameter by using a second precision type conversion algorithm to obtain a first parameter part using a first number of bits and a second parameter part using a second number of bits includes: According to the calibration bit width, the target space mapping parameter is subjected to bit reduction separation processing according to a second predetermined rule to obtain a first parameter part using a first number of bits and a second parameter part using a second number of bits.
7. The method for implementing a grid generator based on a low-precision platform as claimed in claim 6, It is characterized in that The step of performing bit reduction separation processing on the target space mapping parameter according to the calibration bit width and a second predetermined rule to obtain a first parameter part using a first bit number and a second parameter part using a second bit number includes: According to the calibration bit width, obtaining a first parameter part of a corresponding number of bits from the target space mapping parameter stored in the scalar register to obtain a first parameter part using a first number of bits; According to the calibration bit width, the target space mapping parameter stored in the scalar register is shifted right by a corresponding number of bits; According to the calibration bit width, a second parameter part of a corresponding number of bits is obtained from the target mapping parameter after being shifted rightward by a corresponding number of bits in the scalar register, to obtain a second parameter part using a second number of bits.
8. The method for implementing a grid generator based on a low-precision platform as claimed in claim 1, It is characterized in that The step of calculating the target mapping coordinates output by the grid generator according to the first parameter part and the second parameter part by using the first mapping coordinate calculation formula includes: According to the first parameter part and the second parameter part, corresponding parameters in the first mapping coordinate calculation formula are deformed to obtain a second mapping coordinate calculation formula; Initializing the second mapping coordinate calculation formula to obtain an initialization coordinate calculation formula; Calculate the first mapping coordinates according to the initialization coordinate calculation formula; Update the first mapping coordinates using an x-axis coordinate update formula and a y-axis coordinate update formula; The target mapping coordinates output by the grid generator are determined according to the obtained first mapping coordinates by using the vector register and the coordinate and weight calculation formula.
9. A device for implementing a grid generator based on a low-precision platform, It is characterized in that The device comprises: An original space mapping parameter acquisition unit, used for acquiring original space mapping parameters output by a custom network based on the image data when detecting that image data is input into the space transformation network; A parameter fixed-point processing unit, used to perform fixed-point processing on the original space mapping parameters and the specified parameters through a first precision type conversion algorithm to obtain fixed-point scalar data and specified fixed-point numbers, wherein the specified parameters are constants specified in the space transformation formula used in the grid generator; a target space mapping parameter calculation unit, configured to calculate the target space mapping parameters according to the fixed-point scalar data and the specified fixed-point number by using a space transformation formula used in the grid generator; a parameter separation unit, configured to perform a bit reduction separation process on the target space mapping parameter by a second precision type conversion algorithm to obtain a first parameter part using a first number of bits and a second parameter part using a second number of bits, wherein the first number of bits is lower than the second number of bits; A target mapping coordinate calculation unit is used to calculate the target mapping coordinates output by the grid generator according to the first parameter part and the second parameter part through a first mapping coordinate calculation formula.
10. A device for implementing a grid generator based on a low-precision platform, It is characterized in that The method comprises a processor, a register, a memory and a computer program stored in the memory and executable on the processor, wherein the register and / or the processor implements the method according to any one of claims 1 to 8 when executing the computer program.
11. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.
