Data processing method, device and equipment

Abstract

The invention provides a data processing method, device and equipment. The method comprises the steps that a multiplicand and a multiplier to be subjected to product operation are obtained, the digit of the multiplicand is M, and the digit of the multiplier is N; judging whether the multiplier is greater than a preset threshold; if yes, taking the low n-bit data of the multiplier as a target multiplier; if yes, taking data formed by the preset a bit to the (N-1) bit of the multiplier as a target multiplier; wherein N is equal to n + a; inputting the multiplicand and the target multiplier into a multiplier with the bit width of M * n so as to perform multiplication operation on the multiplicand and the target multiplier to obtain a product result; the product result is shifted rightwards by b bits, overflow judgment is carried out on the product result after the product result is shifted rightwards by the b bits, a target product is obtained, and b is the difference value between the preset decimal digits of the multiplier and a. According to the method, the multiplier is shifted, so that the number of bits of the multiplier is reduced, the bit width of the multiplier is further reduced, the area of the multiplier is reduced, and the area of the image signal processor is further reduced.

Description

Technical Field

[0001] The present application relates to the field of semiconductor technology, and in particular to a data processing method, device and equipment. Background Art

[0002] Image Signal Processor (ISP) has been widely used in mobile phones, digital cameras, surveillance cameras, etc. Image signal processors are data stream intensive, characterized by high throughput and concurrent operations, with relatively small control units and more computing units.

[0003] There are multiple algorithms for image signal processors, which involve a large number of integer and decimal multiplications when processing data. In some related technologies, the multiplicand and the multiplier can be input into a multiplier of corresponding bit width for multiplication, and a large number of partial products are generated in the middle, and the partial products are added and carried to obtain the final result.

[0004] Since a multiplier with a larger bit width is needed, the area of ​​the multiplier is larger, which in turn leads to a larger area of ​​the image signal processor. Summary of the invention

[0005] The present application provides a data processing method, device and equipment to solve the problem in the related art that when an image signal processor processes data, a multiplier with a larger bit width is required to perform a multiplication operation on a multiplicand and a multiplier, resulting in a larger area of ​​the multiplier, which in turn results in a larger area of ​​the image signal processor.

[0006] In a first aspect, the present application provides a data processing method, applied to an image signal processor, comprising:

[0007] Get the multiplicand and the multiplier to be multiplied, the number of digits of the multiplicand is M, and the number of digits of the multiplier is N;

[0008] Determine whether the multiplier is greater than a preset threshold; if less than, use the lower n bits of the multiplier as the target multiplier; if greater than, use the data consisting of the preset a-th to N-1-th bits of the multiplier as the target multiplier; wherein N=n+a;

[0009] The multiplicand and the target multiplier are input into a multiplier with a width of M*n to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result;

[0010] The product result is shifted right by b places, and an overflow check is performed on the product result after being shifted right by b places to obtain the target product, where b is the difference between the preset number of decimal places of the multiplier and a.

[0011] In a second aspect, the present application provides a data processing device, applied to an image signal processor, comprising:

[0012] An acquisition module is used to acquire a multiplicand and a multiplier to be multiplied, where the number of digits of the multiplicand is M and the number of digits of the multiplier is N;

[0013] The processing module is used to determine whether the multiplier is greater than a preset threshold value; if less than, the lower n bits of the multiplier are used as the target multiplier; if greater than, the data consisting of the preset a-th to N-1-th bits of the multiplier are used as the target multiplier; wherein N=n+a;

[0014] The processing module is further used to input the multiplicand and the target multiplier into a multiplier with an M*n bit width to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result;

[0015] The processing module is also used to right shift the product result by b bits and perform overflow judgment on the product result after right shifting by b bits to obtain the target product, where b is the difference between the preset decimal places of the multiplier and a.

[0016] In a third aspect, the present application provides a data processing device, applied to an image signal processor, comprising: a processor, a memory, the memory storing codes, the processor running the codes stored in the memory to execute any method as in the first aspect.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement any method as in the first aspect.

[0018] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements any one of the methods in the first aspect when executed by a processor.

[0019] The present application provides a data processing method, device and equipment, which obtains a multiplicand and a multiplier to be multiplied, wherein the number of bits of the multiplicand is M and the number of bits of the multiplier is N. It is judged whether the multiplier is greater than a preset threshold value; if less than, the lower n bits of the multiplier are used as the target multiplier; if greater than, the data consisting of the ath to N-1th bits preset by the multiplier are used as the target multiplier, thereby realizing the right shift of the multiplier and reducing the number of bits of the multiplier from N to n. The multiplicand and the target multiplier are input into a multiplier with a width of M*n to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result. The product result is right-shifted by b bits, and an overflow judgment is performed on the product result after the right shift by b bits to obtain a target product, wherein b is the difference between the number of decimal places preset by the multiplier and a. The present application reduces the number of bits of the multiplier by preferentially right-shifting the multiplier, so that only a multiplier with a smaller bit width is needed to implement the multiplication operation of the multiplicand and the multiplier, thereby reducing the area of ​​the multiplier and further reducing the area of ​​the image signal processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] Figure 1 A schematic diagram of a data processing scenario provided in an embodiment of the present application;

[0022] Figure 2 A flow chart of a data processing method provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of data processing using a prior art technology provided in an embodiment of the present application;

[0024] Figure 4 A flow chart of a method for determining the number of multiplier shift bits a provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of logarithmic fitting of precision error values ​​provided in an embodiment of the present application;

[0026] Figure 6 A flow chart of a method for compensating for loss of a product result provided in an embodiment of the present application;

[0027] Figure 7 A flow chart of a data processing method for a white balance-gain algorithm provided in an embodiment of the present application;

[0028] Figure 8 A module framework diagram of a white balance-gain algorithm provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of comparison of images before and after optimization of a data processing process provided in an embodiment of the present application;

[0030] Fig.10 A schematic diagram of a data processing device provided in an embodiment of the present application;

[0031] Fig.11 A schematic diagram of a data processing device provided in an embodiment of the present application.

[0032] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0034] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0035] Image Signal Processor (ISP) is widely used in mobile phones, digital cameras, surveillance cameras, etc. However, the area of ​​ISP is usually very large, so how to optimize the area of ​​ISP is an important issue.

[0036] Image signal processors are characterized by high throughput and concurrent operations. They are data-intensive, with relatively small control units and more arithmetic units. There are multiple operators in the ISP, and almost every algorithm requires multiplication and addition logic. As a unit that occupies a large area, it is necessary to reduce the area and make a trade-off between hardware cost and performance.

[0037] Since there are multiple algorithms in the image signal processor, involving a large number of integer and decimal multiplications, there are many multiplication and division units with different bit widths. In some related technologies, when many algorithms are implemented in digital circuits, considering the rounding error of fixed-point numbers, in order to ensure data accuracy, the data bit width of the intermediate process will be greater than the data bit width of the final result, that is, multiple multiplication and division units, whose data bit width will be greater than the data bit width of the final result. Specifically, the original values ​​of the multiplier and the multiplicand are input into the multiplier of the corresponding bit width for multiplication. Since a large number of partial products are generated in the intermediate process, the partial products need to be added and carried to obtain the final result, so a multiplier with a large bit width is needed, and finally the actual value is obtained by shifting. Due to the large area of ​​the multiplier, the area of ​​the image signal processor is also large.

[0038] Based on the above problems, in order to reduce the area of ​​the image signal processor, it can be achieved by reducing the area of ​​the multiplier. The area size of the multiplier is related to the following factors:

[0039] 1. The number of bits: The number of bits of a multiplier refers to the number of binary bits of the multiplier and the multiplicand. Generally, the larger the product of the number of bits, the larger the area of ​​the multiplier.

[0040] 2. Precision: The precision of a multiplier refers to the number of significant digits of the product. Generally, the higher the precision, the larger the area of ​​the multiplier.

[0041] 3. Structure: The area of ​​a multiplier varies depending on its structure. For example, the area of ​​an array multiplier is relatively small, while the area of ​​a serial multiplier is relatively large.

[0042] 4. Circuit design: The quality and optimization degree of circuit design will also affect the area of ​​the multiplier. Optimized circuit design can reduce the area of ​​the circuit.

[0043] Therefore, the above factors can be considered comprehensively to select the appropriate multiplier structure and design method to achieve a smaller area and higher performance. In general, multipliers can be directly processed by synthesis tools or call ready-made IPs from EDA (Electronic design automation) manufacturers, which are fast, portable and reliable. Therefore, there is no need to consider the structure and circuit design of the multiplier in particular, and the focus is mainly on the impact of the number of bits and precision of the multiplier on the area.

[0044] Normally, the pixel brightness value of the image finally presented by the image signal processor is in the range of 0-255. Therefore, in the early processing, the high-precision multiplier will occupy more area and function, and the result will fall within a specific bit width, which is less cost-effective. For mid-range image signal processors, based on application scenarios and requirements, area and power consumption are more important factors to consider. Therefore, if the discarded precision is within the acceptable range and the presented image effect is hardly affected, some precision can be discarded to reduce area and power consumption, so as to achieve a better balance between performance and power consumption and improve the cost-effectiveness of the image signal processor.

[0045] Therefore, based on the above factors, for algorithms containing fixed-point multiplication, when the area of ​​the fixed-point multiplier or divider is relatively large, the present application provides a data processing method for use in an image signal processor. Specifically, the judgment logic can be introduced to first right-shift the multiplier to reduce the number of bits of the multiplier. Accordingly, a multiplier with a smaller bit width can be used to implement the multiplication of the multiplicand and the multiplier, thereby abandoning part of the precision, solving the problem of excessive area occupied by the decimal multiplier and divider operation circuit in the hardware circuit, reducing the area of ​​the multiplier, and thus effectively reducing the area of ​​the entire image signal processor.

[0046] Figure 1 A schematic diagram of a data processing scenario provided in an embodiment of the present application, such as Figure 1As shown in the figure, the number of bits of the multiplicand [11:0] is 12 bits, and the number of bits of the multiplier [11:0] is 12 bits. If the multiplier is less than 1024, the lower 10 bits of the multiplier [9:0] are used as the target multiplier; if the multiplier is greater than or equal to 1024, the multiplier [11:2] is used as the target multiplier. The multiplicand and the target multiplier are input to the 12*10-bit wide multiplier to obtain the product result. The number of bits of the product result is 22 bits, which can be recorded as [21:0]. The product result is right-shifted 8 bits again, and the target result obtained can be recorded as [13:0].

[0047] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0048] Figure 2 The flowchart of a data processing method provided by an embodiment of the present application. The method of this embodiment can be executed by an image signal processor and can be implemented by hardware, software, or a combination of hardware and software. Figure 2 As shown, the method may include:

[0049] S201: Obtain a multiplicand and a multiplier to be multiplied, where the number of digits of the multiplicand is M and the number of digits of the multiplier is N.

[0050] In some embodiments, there are multiple algorithms in the image signal processor, including but not limited to white balance-gain algorithm (wb_gain algorithm), automatic exposure, automatic focus and other adjustment algorithms. Each algorithm can be executed based on multiple multipliers, and for different algorithms, the meanings of the data processed, i.e., the multiplicand and the multiplier, also differ to a certain extent.

[0051] For example, the white balance-gain algorithm is used to illustrate that the white balance-gain algorithm is a relatively basic adjustment algorithm used to adjust the color temperature and hue of an image to make it closer to the real color. In the process of executing the white balance-gain algorithm, the gray value can be used as the multiplicand and the corresponding gain value can be used as the multiplier.

[0052] In some embodiments, the number of bits of the multiplicand is M, including the 0th bit to the M-1th bit, which can be recorded as [M-1:0]; the number of bits of the multiplier is N, including the 0th bit to the N-1th bit, which can be recorded as [N-1:0]. The multiplicand and the multiplier are usually represented by binary numbers.

[0053] In one implementation scenario, the number of bits of the multiplicand and the multiplier may be the same or different. Generally, the larger the number of bits of the multiplicand and the multiplier, the larger the bit width of the multiplier is needed to perform the multiplication operation, and the calculation accuracy is relatively high.

[0054] S202: Determine whether the multiplier is greater than a preset threshold; if so, use the lower n bits of the multiplier as the target multiplier; if so, use the data consisting of the preset ath to N-1th bits of the multiplier as the target multiplier; wherein N=n+a.

[0055] In some embodiments, when determining whether the multiplier is greater than a preset threshold, a judgment logic may be performed. Since the hardware part adds a judgment statement without a register, it is equivalent to adding a two-choice data selector, and the area is almost negligible.

[0056] In one implementation scenario, the lower n bits of the multiplier, i.e., [n-1:0], are used as the target multiplier, or the preset a-th to N-1-th bits of the multiplier, i.e., [N-1:a], are used as the target multiplier, which is equivalent to shifting the multiplier right by a bits to obtain the target multiplier. In this case, the number of bits of the target multiplier is n.

[0057] For example, the number of bits of the multiplicand M=12, the number of bits of the multiplier N=12, and the preset number of bits a for right shifting the multiplier is set to 2. At this time, the number of bits of the target multiplier n=10, so the corresponding multiplier bit width is 12*10. At this time, the preset threshold can be set to 2^10=1024.

[0058] In one implementation scenario, if the multiplier is less than 1024, it indicates that the 10th bit and the 11th bit of the multiplier are both 0, and thus the lower 10 bits of the multiplier [9:0] may be taken as the target multiplier.

[0059] In another implementation scenario, if the multiplier is greater than 1024, it indicates that the 10th bit of the multiplier is 1, and therefore [11:2] of the multiplier can be taken as the target multiplier.

[0060] S203: Input the multiplicand and the target multiplier into a multiplier with a width of M*n to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result.

[0061] Since the number of bits of the multiplicand is M and the number of bits of the target multiplier is n, the multiplication operation can be realized by only inputting the multiplicand and the target multiplier into an M*n-bit-width multiplier, without the need for an M*N-bit-width multiplier. Since n<N, the area of ​​the M*n-bit-width multiplier is smaller than that of the M*N-bit-width multiplier, thereby reducing the area of ​​the image signal processor.

[0062] For example, the number of bits of the multiplicand M=12, and the number of bits of the target multiplier n=10, so a 12*10-bit-wide multiplier can implement the multiplication operation of the multiplicand and the target multiplier.

[0063] S204: right-shift the product result by b places, and perform overflow check on the product result after right-shifting by b places to obtain a target product, where b is the difference between the preset number of decimal places of the multiplier and a.

[0064] In some related technologies, the multiplicand and the multiplier are first multiplied, and then the product result is right-shifted based on the preset number of decimal places of the multiplier to obtain the final result. For example, Figure 3 A schematic diagram of data processing using a prior art technology provided in an embodiment of the present application, referring to Figure 3 As shown, the multiplicand number M = 12, recorded as [11:0], and the multiplier number N = 12, recorded as [11:0] are still used for explanation. If the preset decimal place of the multiplier is 10, the number of places of the product of the multiplicand and the multiplier is 24, recorded as [23:0], and the product result needs to be right-shifted by 10 places to obtain the target result [13:0].

[0065] For this application, since the multiplier has been right-shifted by a bits to obtain the target multiplier, if a=2, based on the preset decimal place of 10 bits for the multiplier, it is only necessary to right-shift the product result by 10-2=8 bits to obtain the target result. At this time, b=8. For details, please refer to Figure 1 shown.

[0066] In some embodiments, overflow judgment is performed on the product result after right shifting by b bits. When determining the target product, overflow judgment can be performed on the product result after right shifting by b bits based on the pixel threshold; the pixel threshold is 2 x -1, x represents the color bit depth of the image to be processed. If the product result after right shifting by b bits is greater than the pixel threshold, the pixel threshold is used as the target product. If the product result after right shifting by b bits is less than the pixel threshold, the product result after right shifting by b bits is used as the target product.

[0067] In one implementation scenario, the color depth can be 8 bits, 12 bits, 16 bits, etc. Taking 12 bits as an example, when x is 12, the pixel threshold is 2. 12 -1=4095. If the product result after right shifting by b bits is greater than 4095, the screen cannot display pixels greater than 4095, so 4095 can be used as the target product. If the product result after right shifting by b bits is less than 4095, the original value of the product result after right shifting by b bits is used as the target product.

[0068] The present application provides a data processing method, which obtains a multiplicand and a multiplier to be multiplied, wherein the number of bits of the multiplicand is M and the number of bits of the multiplier is N. It is determined whether the multiplier is greater than a preset threshold value. If it is less than, the lower n bits of the multiplier are used as the target multiplier. If it is greater than, the data composed of the ath to N-1th bits preset by the multiplier are used as the target multiplier, wherein N=n+a. The multiplicand and the target multiplier are input into a multiplier with an M*n bit width to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result. The product result is right-shifted by b bits, and an overflow judgment is performed on the product result after the right shift by b bits to obtain a target product, wherein b is the difference between the number of decimal places preset by the multiplier and a. The method of the present application reduces the number of bits of the multiplier by first shifting the multiplier and then performing a product operation with the multiplicand, so that only a multiplier with a smaller bit width is required to implement it, reducing the area of ​​the multiplier, and further reducing the area of ​​the image signal processor.

[0069] In one or more embodiments of the present application, based on the basic multiplier of M*N bit width required to calculate the multiplicand and the multiplier, the optimized elements of the M*n bit width multiplier can be determined, and the optimized elements include M*(Nn) AND gates, Nn half adders and (M-1)*(Nn) adders.

[0070] In one implementation scenario, taking an array multiplier as an example, a basic multiplier with an M*N bit width may include M*N AND gates, N half adders, and (M*NMN) adders, where the adder is a full adder.

[0071] For a multiplier with an M*n bit width, it includes M*n AND gates, n half adders, and (M*nMn) adders.

[0072] In summary, the number of AND gates, half adders, and adders saved by the M*n bit-width multiplier compared to the M*N bit-width basic multiplier can be calculated.

[0073] For example, if the basic multiplier bit width is 12*12, that is, M=12, N=12; the multiplier bit width of the present application is 12*10, that is, M=12, n=10. Based on the above formula, for a single multiplier, 22 adders, 2 half adders and 24 AND gates can be saved.

[0074] At the same time, since the number of bits of the target multiplier is smaller, the number of bits of the product of the multiplicand and the target multiplier is also reduced accordingly, that is, the bit width occupied by the target multiplier and product registers becomes smaller, and the precision becomes smaller, so the register will also save a certain amount of area.

[0075] In summary, compared with the prior art that uses an M*N-bit width basic multiplier to perform multiplication operations on the multiplicand and the multiplier and shifts the product, the present application first partially shifts the multiplier to obtain an n-bit width target multiplier, and then uses an M*n-bit width multiplier to implement the product operation of the multiplicand and the target multiplier, and then shifts the product result. Compared with the M*N-bit width basic multiplier, the M*n-bit width multiplier reduces resources such as M*(Nn) AND gates, Nn half adders, and (M-1)*(Nn) adders, effectively reducing the area of ​​a single multiplier, and thus effectively reducing the area of ​​the image signal processor.

[0076] Figure 4 A flow chart of a method for determining the number of multiplier shift bits a provided in an embodiment of the present application. In one or more embodiments of the present application, the process of determining a may specifically include: Figure 4 Steps shown:

[0077] S401: Shift the multiplier right by one bit in sequence, calculate the product of the multiplier and the multiplicand after each right shift by one bit, and shift each product right by q bits, where q is the difference between the preset number of decimal places of the multiplier and the number of bits of the current right shift of the multiplier.

[0078] In some embodiments, the number of bits of the current multiplier shifted right can be recorded as p, and the number of decimal places preset for the multiplier is set to s, then q=sp.

[0079] For example, if M=12, N=12, and the preset number of decimal places of the multiplier is s=10, the multiplier is right-shifted by 2 places, ie, p=2, and therefore the product needs to be right-shifted by 10-2=8 places, ie, q=8.

[0080] S402: Calculate the difference between each product after right shift by q bits and the basic product result, and determine whether the ratio of the difference to the pixel threshold is less than the number of shift bits corresponding to the error threshold.

[0081] In some embodiments, due to the first right shift of the multiplier, the number of bits of the multiplier is reduced, which will cause a certain precision loss, that is, the difference between each product and the basic product result.

[0082] In one implementation scenario, let the multiplicand multiplier Where M is the number of digits of the multiplicand x, N is the number of digits of the multiplier y, and x i Can be 0 or 1, y i Can be 0 or 1.

[0083] For the above multiplicand and multiplier, two methods can be used to calculate the product, as follows:

[0084] The first method is that the prior art uses an M*N multiplier to first multiply x and y and then shift to obtain a basic product result.

[0085] The second method is to use the improved small bit width multiplier of the present application, that is, the M*n bit width multiplier for calculation. Specifically, the multiplier is first shifted, the shifted multiplier is multiplied with the multiplicand to obtain the corresponding multiplier, and then the product is shifted.

[0086] Still taking M=12, N=12, s=14 as an example, the number of bits of the product of x and y is 24 bits, so the number of bits of the product of x and y right shifted based on the first method is 10 bits. The number of bits of the second method to shift the multiplier is still recorded as p, and the number of bits to shift the product is still recorded as q, and p+q=10.

[0087] Specifically, taking the multiplier being shifted by 2 bits first, that is, p=2 as an example, the difference f between the result calculated by the second method and the result calculated by the first method can be expressed by the following formula:

[0088] f=y1(x M-1 x M-2 x M-3 )+y0(x M-1 x M-2 ) (1)

[0089] That is: f=y1(x[M-1:M-3])+y0(x[M-1:M-2])(2)

[0090] In the above formula, x M-1 Represents the highest digit value of the multiplicand x, x M-2 Represents the second highest value of the multiplicand x, x M-3 represents the third highest value of the multiplicand x, y0 represents the lowest value of the multiplier y, y1 represents the second lowest value of the multiplier y, and M represents the number of bits of the multiplicand x. In this example, M=12.

[0091] From the above formula, we can know that the difference f is related to the upper three digits of the multiplicand and the lower two digits of the multiplier. When the upper three digits of the multiplicand and the lower two digits of the multiplier are all 1, the highest precision loss corresponds to the difference f. When the upper three digits of the multiplicand are all 0, f = 0, indicating that there is no error; when the lower two digits of the multiplier are all 0, f is also 0, which still indicates that there is no error.

[0092] Specifically, the three upper bits of the multiplicand are all 0, indicating that the 9th, 10th and 11th bits of the multiplicand are all 0. Therefore, it can be expressed that when the multiplicand is less than 512, there is no error in the calculation using the second method of the present application.

[0093] In one implementation scenario, when the multiplicand x is greater than 512, the precision loss, i.e., the difference f, increases with the value of the multiplier y%4, i.e., increases by 1-3, with the maximum precision loss being 11, every time the multiplicand x doubles.

[0094] In the above, y%4 represents the remainder when y is divided by 4. When y%4=0, it means y0=0, y1=0. At this time, f=0, indicating that there is no error; when y%4=1, it means y0=1, y1=0; when y%4=2, it means y0=0, y1=1; when y%4=3, it means y0=1, y1=1.

[0095] In general, in image signal processors, the high-precision image data of interest is usually the image brightness, and the image brightness perceived by the human eye is in a logarithmic relationship with the increase of image brightness. The loss of precision is also in a logarithmic relationship with the increase of brightness. For details, please refer to Figure 5 shown.

[0096] Figure 5 A schematic diagram of logarithmic fitting of accuracy error values ​​provided in an embodiment of the present application, wherein curve l0 corresponds to y%4=0, a situation where there is no error; curve l1 corresponds to the error situation at different brightnesses when y%4=1; curve l2 corresponds to the error situation at different brightnesses when y%4=2; curve l3 corresponds to the error situation at different brightnesses when y%4=3.

[0097] Through calculation, we know that the maximum error rate is 11 / 4095≈0.27%<0.3%, and it is in a logarithmic relationship. Therefore, for general image signal processors, this part of the precision loss conforms to the law of human eyes, so it can be judged that this error can be discarded. At this time, the bit width of a single multiplier operation is reduced, the area is reduced, and the power consumption is also relatively reduced. Among them, 4095 is the pixel threshold corresponding to the image color bit depth of 12 bits.

[0098] S403: If there are multiple shift bit numbers, the maximum value of the shift bit numbers is taken as a.

[0099] The number of shift bits of the multiplier, i.e. the value of a, will affect the number of bits of the multiplier, and thus the precision of the product. When the value of a is larger, the number of bits of the multiplier is smaller, and the precision of the product is relatively lower. Correspondingly, the area of ​​the corresponding multiplier will also be smaller. Therefore, the area of ​​the multiplier can be minimized while meeting the precision requirements.

[0100] For example, if the number of shift bits is 1 and 2 respectively, the corresponding products of the multiplier and the multiplicand are right-shifted accordingly, and the ratio of the difference between the product after right shift and the basic product result and the pixel threshold is less than the error threshold, which means that the difference can be ignored. Therefore, the maximum value of 2 can be taken as the a value to minimize the number of bits of the multiplier to the greatest extent while ensuring a certain accuracy, so as to reduce the number of bits of the corresponding multiplier and thus reduce the area of ​​the multiplier.

[0101] In summary, by shifting the multiplier by different numbers of bits, calculating the product of the shifted multiplier and the multiplicand, and right-shifting the product, and calculating the difference between the right-shifted product and the basic product result, the number of shifted bits corresponding to the ratio of the difference to the pixel threshold is obtained. The maximum value of the multiple shifted bits is used as the shifted bit number of the multiplier, i.e., the value a, so that the determined value a can reduce the area of ​​the multiplier to a large extent under the condition of a certain precision loss.

[0102] Figure 6 A flowchart of a method for compensating for loss of a multiplication result provided in an embodiment of the present application. In one or more embodiments of the present application, for an image signal processor with high precision requirements, an adder circuit or the like may be used to compensate for loss of precision. Specifically, the method for compensating for loss of a multiplication result includes: Figure 6 Steps shown:

[0103] S601: Calculate the target difference between the product result after right shifting by b bits and the basic product result according to the multiplicand and the target multiplier. The basic product result is the product of the multiplicand and the multiplier, and right shifted by a+b bits.

[0104] The target difference is the loss of precision of the product result after right shifting by b bits compared to the base product result.

[0105] In one implementation scenario, still taking the case where the number of digits of the multiplicand is M=12, the number of digits of the multiplier is N=12, the number of decimal places of the multiplier is s=10, and a=2 as an example, the number of digits of the target multiplier is n=10, and b=8.

[0106] S602: Using an adder, add the target difference and the product result after right shifting by b bits to compensate for the loss of the product result after right shifting by b bits.

[0107] In one implementation scenario, based on the above formula (2), it can be seen that the lost precision, i.e., the difference f, is mainly related to the upper three bits of the multiplicand and the lower two bits of the multiplier. Therefore, the loss can be compensated by adding a 3-bit wide adder and two half adders.

[0108] Compared with the above embodiment, the basic multiplier bit width is 12*12, that is, M=12, N=12; the multiplier bit width of the present application is 12*10, that is, M=12, n=10. For a single multiplier, 22 adders, 2 half adders and 24 AND gates can be saved. If loss compensation is required, the multiplier of the present application can save 21 adders and 24 AND gate resources.

[0109] In summary, for image signal processors with high precision requirements, adders and half adders can be used to shift the multiplier first and then compensate for the loss caused by multiplying it with the multiplicand. While achieving high precision requirements, it also reduces the area of ​​the multiplier, thereby reducing the area of ​​the image signal processor.

[0110] The image signal processor includes a white balance-gain algorithm, the multiplicand is a pixel value, and the multiplier is a gain value. Based on the above embodiment, the white balance-gain algorithm included in the image signal processor is described in detail below. For the white balance-gain algorithm, the prior art generally first multiplies two large bit width data, and then according to the position of the decimal point specified by the fixed point number, the calculated product is right-shifted by the corresponding number of decimal places to obtain the final value. This method is simple and direct, but due to the large bit width of the multiplier and the multiplicand, the area resource consumption of the multiplier increases geometrically with the increase of the bit width, and the multiplier occupies a large area.

[0111] The present application first shifts the multiplier to reduce the bit width of the multiplier, and then performs a product operation with the multiplicand, so that a small bit width multiplier can be used, thereby reducing the occupied area resources. The specific implementation process of the white balance-gain algorithm is described in detail below, assuming that both the multiplicand and the multiplier are 12 bits, and the number of bits a=2 is first right-shifted to the multiplier, and the number of decimal places preset for the multiplier is 10 bits.

[0112] Figure 7 A flow chart of a data processing method of a white balance-gain algorithm provided in an embodiment of the present application. The data processing method based on the white balance-gain algorithm can be constructed as follows Figure 8 The module framework of the white balance-gain algorithm shown in FIG. Figure 8 A module framework diagram of a white balance-gain algorithm provided in an embodiment of the present application. Figure 7 and Figure 8 As shown, the method is as follows:

[0113] S701: Acquire image data.

[0114] In an implementation scenario, the image data may be data in a RAW format, and the data may be 12 bits.

[0115] S702: Based on the image data, determine the positions of red and blue through the filter array to determine the pixel value of the red position and the corresponding gain value, and the pixel value of the blue position and the corresponding gain value.

[0116] In some embodiments, the filter array may be a Bayer array, which may use a 1 red, 2 green, 1 blue arrangement to convert grayscale information into color information, or other arrangements, which are not limited in the present application. According to the Bayer pattern on the filter, if it is a green position, the output pixel value is equal to the input pixel value; if it is a red position, the output pixel value is equal to the value after the input pixel value plus the red gain; if it is a blue position, the output pixel value is equal to the value after the input pixel value plus the blue gain. Therefore, there is no need to calculate the pixel value and gain value of the green position.

[0117] In one implementation scenario, the bayer array can be used to determine which chromaticity of the three primary colors of RGB is covered on each pixel value, thereby determining the positions of red and blue.

[0118] In one implementation scenario, the pixel value at the red position is real-time data, and the pixel value may also be called a grayscale value. The gain value rgain at the red position and the gain value bgain at the blue position may be stored in a register.

[0119] S703: Determine whether the gain value of the red position and the gain value of the blue position are less than 1024. If so, execute steps S704 and S706-S707; if not, execute steps S705-S707.

[0120] S704: Take the lower 10 bits of the gain value as the target gain value.

[0121] S705: Take the 2nd to 11th bit data of the multiplier as the target gain value.

[0122] In one implementation scenario, a judgment logic may be made to judge whether the gain value at the red position and the gain value at the blue position are less than 1024. After the judgment logic is made, a 12*10-bit wide multiplier is used to instantiate the system into the white balance-gain algorithm.

[0123] The lower 10 bits of the gain value are data [9:0], and the 2nd to 11th bits of the gain value are data [11:2].

[0124] S706: Based on the grayscale world assumption, the pixel values ​​at the red and blue positions are multiplied by the corresponding target gain values ​​to obtain corresponding product results, and the product results are right-shifted by 8 bits to obtain the target product.

[0125] The real-time pixel values ​​of the red and blue positions are used as multiplicands and input to the multiplicand input port of the multiplier. The gain value rgain of the red position and the gain value bgain of the blue position are used as corresponding multipliers and input to the multiplier input port of the multiplier.

[0126] In some related technologies, both the pixel value and the gain value are 12 bits wide, and finally right-shifted by 10 bits to obtain the final grayscale value, that is, the product result, and the multiplier used is a 12*12-bit wide multiplier.

[0127] The present application preferentially shifts the gain value right by 2 bits, enters the multiplier with the pixel value, and finally obtains the product by shifting right by 8 bits. The multiplier used is a 12*10-bit wide multiplier.

[0128] Taking array multipliers as an example, for a single multiplier, compared with a 12*12-bit multiplier, a 12*10-bit multiplier saves 22 adders, 2 half adders, and 24 AND gates. At the same time, because the bit width of the multiplier and product registers is smaller and the precision is smaller, the registers will also save area. Estimated by the bit width product, it can save about 16% of the area.

[0129] In some embodiments, since the method of the present application is to shift the gain value first and reduce the bit width of the gain value, a certain loss of precision will be caused. According to formula (2) of the above embodiment, it can be seen that the lost precision is mainly related to the upper three bits of the pixel value and the lower two bits of the gain value. If it is necessary to compensate for the lost precision, it is necessary to add a 3-bit wide adder and 2 half adders for compensation, which can save the resources of 21 adders and 24 AND gates. However, since this part of the lost precision occurs when the gain value is greater than 1024, the human eye has a weak ability to distinguish high brightness, the highest precision loss is 11, the relative error is less than 11 / 4095≈0.27%<0.3%, and it is logarithmic, which conforms to the law of the human eye, so it can be judged that this part of the precision loss can be ignored.

[0130] S707: Perform overflow processing on the target product and output it.

[0131] Overflow processing is performed on the target product to obtain a true color close to the human eye. In one implementation scenario, if the target product is greater than 4095, the maximum value 4095 is taken. If the target product is less than 4095, the original value is taken.

[0132] In some embodiments, in order to test the resource saving achieved by the data processing method of the present application and the effect of the presented image, the hardware code was written according to the above steps and synthesized using a synthesis tool, and the area of ​​the multiplier before optimization was 896um. 2 , the optimized multiplier area is 772um 2 , saving more than 100um2 , on the original basis, about 14% of the area is saved. Among them, before optimization refers to the use of existing technology based on large bit width multipliers for data processing, and after optimization refers to the use of the application based on small bit width multipliers for data processing.

[0133] A comparative test was also conducted on the imaging effects of the images before and after optimization. The parameters used included: the input pixel value was a gradient value between [1024:4095], and the gain value was selected as 1027, which is equivalent to the case with the most serious precision loss. Fig. 9 A schematic diagram showing a comparison of images before and after optimization of a data processing process is shown in FIG. Fig. 9 As shown, the left picture is the image obtained based on the large bit-width multiplier before optimization, and the right picture is the image obtained based on the small bit-width multiplier after optimization. There is no obvious difference between the two in human vision.

[0134] In summary, for the white balance-gain algorithm, the gain value as the multiplier is first shifted right by 2 bits to reduce the bit width of the gain value, so that a multiplier with a small bit width can perform a product operation on the pixel value and the gain value to achieve data processing. Since the bit width of the multiplier used is reduced, its area is also reduced, thereby reducing the area of ​​the image signal processor.

[0135] It should be noted that the above embodiment is only described with the white balance-gain algorithm. Since the image signal processor includes multiple algorithms, the specific situation can be combined with the image signal processor algorithm. The data processing method of the present application can be applied to multiple algorithms that require multiplication operations, and there is no restriction on the algorithm type.

[0136] Fig.10 A schematic diagram of a data processing device provided in an embodiment of the present application. Fig.10 As shown, an embodiment of the present application provides a data processing device 1000 , which is applied to an image signal processor and may include an acquisition module 1001 and a processing module 1002 .

[0137] An acquisition module 1001 is used to acquire a multiplicand and a multiplier to be multiplied, where the number of digits of the multiplicand is M and the number of digits of the multiplier is N;

[0138] The processing module 1002 is used to determine whether the multiplier is greater than a preset threshold value; if less than, the lower n bits of the multiplier are used as the target multiplier; if greater than, the data consisting of the preset a-th to N-1-th bits of the multiplier are used as the target multiplier; wherein N=n+a;

[0139] The processing module 1002 is further used to input the multiplicand and the target multiplier into a multiplier with an M*n bit width to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result;

[0140] The processing module 1002 is further used to right shift the product result by b bits and perform overflow judgment on the product result after right shifting by b bits to obtain a target product, where b is the difference between the preset decimal places of the multiplier and a.

[0141] The device of this embodiment can be used to execute the method embodiment shown above. Its implementation principle and technical effects are similar and will not be described in detail here.

[0142] Fig.11 Schematic diagram of a data processing device provided in an embodiment of the present application, the data processing device is applied to an image signal processor. Fig.11 As shown, an embodiment of the present application provides a data processing device 1100 including a processor 1101 and a memory 1102 , wherein the processor 1101 and the memory 1102 are connected via a bus 1103 .

[0143] In the specific implementation process, the memory 1102 stores hardware codes, and the processor 1101 runs the hardware codes stored in the memory 1102 to execute the method of the above method embodiment.

[0144] The specific implementation process of the processor 1101 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0145] In the above Fig.11 In the illustrated embodiment, it should be understood that the processor 1101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), digital image processors, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the invention may be directly embodied as being executed by a hardware processor.

[0146] The memory 1102 may include a high-speed RAM memory, and may also include a static random access memory (SRAM) and the like.

[0147] The bus 1103 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 1103 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus 1103 in the drawings of the present application is not limited to only one bus or one type of bus.

[0148] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method of the above method embodiment.

[0149] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0150] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0151] An embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the method provided in any embodiment of the present application is implemented.

[0152] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0153] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data processing method, characterized in that: Applications in image signal processors, including: Obtain a multiplicand and a multiplier to be multiplied, wherein the number of digits of the multiplicand is M and the number of digits of the multiplier is N; Determine whether the multiplier is greater than a preset threshold; if less than, use the lower n bits of the multiplier as the target multiplier; if greater than, use the data consisting of the preset ath to N-1th bits of the multiplier as the target multiplier; wherein N=n+a; Inputting the multiplicand and the target multiplier into a multiplier with an M*n bit width to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result; The product result is right-shifted by b places, and an overflow check is performed on the product result after right-shifting by b places to obtain a target product, where b is the difference between the preset number of decimal places of the multiplier and a.

2. The method according to claim 1, characterized in that The overflow judgment is performed on the product result after right shifting by b bits to obtain the target product, including: Based on the pixel threshold, overflow judgment is performed on the product result after right shifting by b bits; the pixel threshold is 2 x -1, where x represents the value of the color depth of the image to be processed; If the product result after right shifting by b bits is greater than the pixel threshold, the pixel threshold is used as the target product; If the product result after right shifting by b bits is less than the pixel threshold, the product result after right shifting by b bits is used as the target product.

3. The method according to claim 2, characterized in that The method further comprises: According to the multiplicand and the target multiplier, calculate a target difference between a product result after right shift by b bits and a basic product result, wherein the basic product result is a value of the product of the multiplicand and the multiplier right shifted by a+b bits; An adder is used to add the target difference and the product result after right shifting by b bits, so as to compensate for the loss of the product result after right shifting by b bits.

4. The method according to claim 3, characterized in that: The method further comprises: Shift the multiplier right by one bit in sequence, calculate the product of the multiplier and the multiplicand after each right shift by one bit, and shift each product right by q bits, where q is the difference between the number of decimal places preset for the multiplier and the number of places shifted right by the current multiplier; Calculate the difference between each product after right shift by q bits and the basic product result, and determine that the ratio of the difference to the pixel threshold is less than the number of shift bits corresponding to the error threshold; If there are multiple shift bits, the maximum value among the shift bits is taken as a.

5. The method according to claim 1 or 2, characterized in that: The multiplier is an array multiplier; the method further comprises: Based on the M*N bit-width basic multiplier required to calculate the multiplicand and the multiplier, the optimized elements of the M*n bit-width multiplier are determined, and the optimized elements include M*(Nn) AND gates, Nn half adders and (M-1)*(Nn) adders.

6. The method according to claim 1, characterized in that The image signal processor includes a white balance-gain algorithm, the multiplicand is a pixel value, and the multiplier is a gain value; The step of obtaining a multiplicand and a multiplier to be multiplied includes: Get image data; Based on the image data, the positions of red and blue are determined by a filter array to determine a pixel value at the red position and a corresponding gain value, and a pixel value at the blue position and a corresponding gain value.

7. A data processing device, characterized in that: Applications in image signal processors, including: An acquisition module, used for acquiring a multiplicand and a multiplier to be multiplied, wherein the number of digits of the multiplicand is M and the number of digits of the multiplier is N; A processing module, used to determine whether the multiplier is greater than a preset threshold; if less than, use the lower n bits of the multiplier as the target multiplier; if greater than, use the data consisting of the preset a-th to N-1-th bits of the multiplier as the target multiplier; wherein N=n+a; The processing module is further used to input the multiplicand and the target multiplier into a multiplier with an M*n bit width to perform a multiplication operation on the multiplicand and the target multiplier to obtain a product result; The processing module is also used to right shift the product result by b bits and perform overflow judgment on the product result after right shifting by b bits to obtain a target product, where b is the difference between the preset number of decimal places of the multiplier and a.

8. A data processing device, applied to an image signal processor, comprising: A processor and a memory, wherein the memory stores hardware codes, and the processor runs the hardware codes stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images