Method and apparatus for processing an image

Abstract

The application provides a method and device for processing an image. The method comprises: obtaining image data; calculating filter weight coefficients of the image data by using a first processing unit and a second processing unit, wherein the filter weight coefficients of the image data comprise filter weight coefficients corresponding to first data elements and filter weight coefficients corresponding to second data elements, the filter weight coefficients corresponding to the first data elements and the filter weight coefficients corresponding to the second data elements comprise a first part with the same value and a second part with different values, the first part is calculated by the first processing unit, and the second part is calculated by the second processing unit, wherein the first processing unit is a vector processor, and the second processing unit is a vector processor or a scalar processor. The application can use the first processing unit to perform the calculation of the data with the same value in the filter weight coefficients, so that the part that needs to be repeatedly calculated only needs to be calculated once, the amount of calculation is reduced, and the operation efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of image data processing, and specifically to a method and apparatus for processing images. Background Technology

[0002] Image filtering is widely used in computer vision. Image filtering generally requires vector-level operations on image data, which results in problems such as large computational load and low computational efficiency. Summary of the Invention

[0003] This application provides a method and apparatus for processing images. The various aspects of this application will be described below.

[0004] In a first aspect, a method for processing an image is provided, the method comprising: acquiring image data; calculating filter weight coefficients of the image data using a first processing unit and a second processing unit, wherein the filter weight coefficients of the image data include filter weight coefficients corresponding to a first data element and filter weight coefficients corresponding to a second data element, the filter weight coefficients corresponding to the first data element and the filter weight coefficients corresponding to the second data element include a first part having the same value and a second part having different values, the first part being calculated by the first processing unit and the second part being calculated by the second processing unit, wherein the first processing unit is a vector processor and the second processing unit is a vector processor or a scalar processor.

[0005] In a second aspect, an image processing apparatus is provided, the apparatus comprising: an acquisition unit configured to acquire image data; and a calculation unit configured to calculate filter weight coefficients of the image data using a first processing unit and a second processing unit, wherein the filter weight coefficients of the image data include filter weight coefficients corresponding to a first data element and filter weight coefficients corresponding to a second data element, the filter weight coefficients corresponding to the first data element and the filter weight coefficients corresponding to the second data element include a first part having the same value and a second part having different values, the first part being calculated by the first processing unit and the second part being calculated by the second processing unit, wherein the first processing unit is a vector processor and the second processing unit is a vector processor or a scalar processor.

[0006] Thirdly, a computer program product is provided, the computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the above aspects.

[0007] Fourthly, a computer-readable medium is provided that stores program code, which, when run on a computer, causes the computer to perform the methods described in the preceding aspects.

[0008] This application utilizes the first processing unit to perform calculations on the same data values ​​in the filter weight coefficients, so that parts that originally required repeated calculations only need to be calculated once, thereby reducing the amount of computation and improving computational efficiency. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of an image data vector processing process provided in an embodiment of this application.

[0010] Figure 2 This is a schematic diagram of data used to calculate the weight coefficients of Vw(3,2;2,2), Vw(2,2;3,2), Vw(4,2;2,2) and Vw(2,2;4,2) according to an embodiment of this application.

[0011] Figure 3 This is a schematic diagram of data used to calculate the weight coefficients of Vw(2,0;2,2) and Vw(2,4;2,2), Vw(2,1;2,2) and Vw(2,3;2,2) according to embodiments of this application.

[0012] Figure 4 This application provides a schematic diagram of data used to calculate the weighting coefficients of Vw(3,0; 2,2) and Vw(2,4; 3,2), Vw(3,1; 2,2) and Vw(2,3; 3,2) in embodiments of this application.

[0013] Figure 5 A schematic diagram of data for calculating the weighting coefficients of Vw(4,0; 2,2) and Vw(2,4; 4,2), Vw(4,1; 2,2) and Vw(2,4; 4,2) provided in the embodiments of this application.

[0014] Figure 6 This is a flowchart illustrating the image processing method provided in an embodiment of this application.

[0015] Figure 7 A schematic diagram illustrating the calculation process of Vw(2,0;2,2) and Vw(2,4;2,2), Vw(2,1;2,2) and Vw(2,3;2,2) provided for embodiments of this application.

[0016] Figure 8 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0018] Image processing uses computers to analyze and process images to achieve desired data results. Image processing generally refers to the processing of digital images, which are images that exist in their natural form but have been transformed into a digital form suitable for computer processing. Internally, an image can be represented as a digital array, where each element can be a pixel.

[0019] Image filtering is widely used in computer vision (CV), and common filtering methods include median filtering, Gaussian filtering, and surface filtering. Surface filtering is generally used for image smoothing and is a relatively important image filtering algorithm. The following is an exemplary introduction to surface filtering algorithms.

[0020] Taking a filtering size of 5×5 as an example, the surface filtering algorithm can process the input point to be processed, and then calculate the distance of the data points within the 5×5 square frame centered on the input point to the center point. Based on the distance, the filtering weight coefficient w(k,l,i,j) of the data within the filtering size on the input data can be calculated, where w(k,l,i,j)=max(0,255―|x i+k,j+l -x i,j |·6). The values ​​of k and l are determined by the filter size. For example, when the filter size is 7×7, the range of k and l is [-3, +3], and when the filter size is 9×9, the range of k and l is [-4, +4]. i and j are the coordinates of the data in the image. For example, the coordinates of the input data can be (0,0), meaning the input point can be set as the origin. In other words, based on the above calculation formula, the filter weight coefficient w(k,l,i,j) of all data within the filter size range centered on the output point can be calculated. Then, the cumulative value sum_weight of all 25 filter weight coefficients is calculated. The cumulative value of the filter weight coefficients can be represented by sw(i,j). The filter weight coefficients of the 25 data points are multiplied by the 25 data points themselves, and the cumulative sum is calculated as sum_weight. This cumulative sum can be represented by sx(i,j). The output data y after the input point is surface filtered i,j It can be calculated using the following formula.

[0021]

[0022] After simplification, In other words, for an input data x i,j When performing filtering, it is necessary to use the input data x i,j Using the center point, calculate the n within the filter size n×n. 2The filter weight coefficients w(k,l,i,j) of the input data are calculated, and the cumulative value of the filter weight coefficients and the cumulative value of the product of the coefficients and the data are calculated to obtain the filtered output data y. i,j .

[0023] In image processing, data operations can be performed using vector processors, which can process multiple data sets at once. The following section combines... Figure 1 An exemplary description of the image data vector processing procedure is provided.

[0024] Figure 1 The image shown may include 16×16 data points, which could be, for example, the grayscale values ​​of image pixels. The portion requiring image filtering is an 8×12 data array. Figure 1 middle The filter size is 5×5. Therefore, when calculating the filter weight coefficients using 8×12 data points as the center point, the data used to calculate the filter weight coefficients is a 12×16 data array. Figure 1 middle To facilitate data processing, vector processors can have a Single Instruction Multi-Data (SIMD) setting of 8, i.e., SIMD = 8. A vector processor can process one line of eight data items. For example... Figure 1 In this context, the SIMD instruction can calculate the output data x92 to x99 from the first row of inputs x22 to x29, and then vertically calculate the output data y92 to y99 from the eighth row of inputs x92 to x99. As the center point shifts from y22 to y29, the filter size also shifts accordingly. The following example demonstrates the calculation process using the calculation of the output data x92 to x99 from the first row of inputs x22 to x29.

[0025] Expressed in vector form, denoted as That is, the vector subscript is the subscript of its smallest element.

[0026] Then we have:

[0027]

[0028] Vsw 2，2 =Vw 0，0；2，2 +Vw 0，1；2，2 +Vw 0，2；2，2 +Vw 0，3；2，2 +Vw 0，4；2，2 +Vw 1，0；2，2 +Vw 1，1；2，2 +Vw 1，2；2，2 +Vw 1，3；2，2 +Vw 1，4；2，2 +Vw 2，0；2，2+Vw 2，1；2，2 +Vw 2，2；2，2 +Vw 2，3；2，2 +Vw 2，4；2，2 +Vw 3，0；2，2 +Vw 3，1；2，2 +Vw 3，2；2，2 +Vw 3，3；2，2 +Vw 3，4；2，2 +Vw 4，0；2，2 +Vw 4，1；2，2 +Vw 4，2；2，2 +Vw 4，3；2，2 +Vw 4，4；2，2

[0029] Vsx 2，2 =Vw 0，0；2，2 Vx 0，0 +Vw 0，1；2，2 Vx 0，1 +Vw 0，2；2，2 Vx 0，2 +Vw 0，3；2，2 Vx 0，3 +Vw 0，4；2，2 Vx 0，4 +Vw 1，0；2，2 Vx 1，0 +Vw 1，1；2，2 Vx 1，1 +Vw 1，2；2，2 Vx 1，2 +Vw 1，3；2，2 Vx 1，3 +Vw 1，4；2，2 Vx 1，4 +Vw 2，0；2，2 Vx 2，0 +Vw 2，1；2， 2Vx 2，1 +Vw 2，2；2，2 Vx 2，2 +Vw 2，3；2，2 Vx 2，3 +Vw 2，4；2，2 Vx 2，4 +Vw 3，0；2，2 Vx 3，0 +Vw 3，1；2，2 Vx 3，1 +Vw 3，2；2，2 Vx 3，2 +Vw 3，3；2，2 Vx 3，3 +Vw 3，4；2，2 Vx 3，4 +Vw 4，0；2，2 Vx 4，0 +Vw 4，1；2，2 Vx 4，1 +Vw 4，2；2，2 Vx 4，2 +Vw 4，3；2，2 Vx4，3 +Vw 4，4；2， 2Vx 4，4

[0030] Where w(0, 0; 2, 2) = max(0, 255 - |x 00 -x 22 |·6), w(0,1;2,3)=max(0,255-|x 01 -x 23 |·6), w(0,2;2,4)=max(0,255-|x 02 -x 24 |·6), w(0,3;2,5)=max(0,255-|x 03 -x 25 |·6), w(0,4;2,6)=max(0,255-|x 04 -x 26 |·6), w(0,5;2,7)=max(0,255-|x 05 -x 27 |·6), w(0,6;2,8)=max(0,255-|x 06 -x 28 |·6), w(0,7;2,9)=max(0,255-|x 07 -x 29 |·6) When calculating the filter weight coefficients, it is necessary to load the calculation data and the center point data separately. The calculation data can be the data within the filter size, and the center point data is the data to be processed in the image. During vector-level operations, five sliding window operations need to be performed on the input vector data Vx, i.e., from -2 to +2. In vector processors, this is generally accomplished through unaligned loading or element-level shifting.

[0031] The computational complexity of calculating the weight coefficients for vector-level filtering is extremely high. Figure 1 For example, for an input vector, two vectors need to be loaded, that is, two rows of data need to be loaded, the result of one data operation needs to be saved, 20 data element shift instructions need to be executed, 24 vector operations need to be executed, including vector operations such as calculating the absolute value of the difference (ABSSUB) and the maximum value (MAX), 25 vector additions need to be executed, 25 vector multiplications and accumulations need to be executed, and finally a vector division needs to be executed. The amount of computation is quite considerable.

[0032] The inventors discovered that when calculating the filter weight coefficients w(k, l, i, j), it is necessary to calculate the absolute value of the difference between two data elements, |x|. i+k，j+l -x i，jThe absolute value operation of the difference (ABSSUB) exhibits symmetry. That is, ABSSUBx,y = ABSSUBy,x, therefore, w(k,l;i,j) = max(0,255-|x i+k，j+l -x i，j |·6)=w(i,j;k,l). With Figure 1 For example, in Figure 1 In, w(2,2;3,2)=w(3,2;2,2), w(4,2;22)=w(2,2;2,4).

[0033] Taking the calculation of SIMD=8 vector level filter weight coefficients as an example, Figure 2 The data used to calculate the weighting coefficients of Vw(3,2;2,2), Vw(2,2;3,2), Vw(4,2;2,2), and Vw(2,2;4,2) are listed. Figure 2 As shown in section 2.1, when calculating Vw(3, 2; 2, 2), the absolute values ​​of the differences between x32 and x39 and the center points are calculated using x22 to x29 as center points, in order to calculate the filter weight coefficients w(k, l, i, j). When calculating Vw(2, 2; 3, 2), the absolute values ​​of the differences between x22 and x29 and the center points are calculated using x32 to x39 as center points. However, |x 32～39 -x 22～29 |=|x 32～39 -x 22～29 |, that is Similarly, Vw(4,2;2,2)=Vw(2,2;4,2).

[0034] Furthermore, the inventors discovered that in addition to the completely symmetrical data mentioned above, there are also partially symmetrical data. Figure 3 Several sets of data are provided for calculating the weight coefficients of vector-level filtering. Figure 3 Section 3.1 shows the data used to calculate the weighting coefficients for Vw(2,0; 2,2) and Vw(2,4; 2,2). When calculating Vw(2,0; 2,2), the weighting coefficients of x20–x27 are calculated with respect to the center points x22–x29. Similarly, when calculating Vw(2,4; 2,2), the weighting coefficients of x24–x2b are calculated with respect to the center points x22–x29. Where |x 22～27 -x 24～29 |=|x 24～29 -x 22～27 | That is, when calculating the weight coefficients of Vw(2,0;2,2) and Vw(2,4;2,2), due to the symmetry of the absolute value of the difference, there are 6 identical data elements and 2 different data elements for each. Figure 3 middle Figure 3Figure 2 shows the data used to calculate the weighting coefficients of Vw(2,1; 2,2) and Vw(2,3; 2,2). When calculating Vw(2,1; 2,2), the weighting coefficients of x21 to x28 are calculated with respect to the center points x22 to x29 respectively. Similarly, when calculating Vw(2,3; 2,2), the weighting coefficients of x23 to x2a are calculated with respect to the center points x22 to x29 respectively. Where |x 22～28 -x 23～29 |=|x 23～29 -x 22～28 |, that is, when calculating the weight coefficients of Vw(2,1;2,2) and Vw(2,3;2,2), due to the symmetry of the absolute value of the difference, there are 7 data elements with the same value and 1 data element with a different value.

[0035] by Figure 4 For example, Figure 4 Section 4.1 shows the data used to calculate the weighting coefficients for Vw(3,0; 2,2) and Vw(2,4; 3,2). When calculating Vw(3,0; 2,2), the weighting coefficients of x30–x37 with respect to the center points are calculated, using x22–x29 as the center points. When calculating Vw(2,4; 3,2), the weighting coefficients of x24–x2b with respect to the center points are calculated, using x32–x39 as the center points. Where, |x 32～37 -x 24～29 |=|x 24～29 -x 32～37 | means that each has 2 data elements with different values ​​and 6 data elements with the same value. Figure 4 Section 4.2 shows the data used to calculate the weighting coefficients of Vw(3,1;2,2) and Vw(2,3;3,2). When calculating Vw(3,1;2,2), the weighting coefficients of x31–x38 with respect to the center points are calculated using x22–x29 as the center points. Similarly, when calculating Vw(2,3;3,2), the weighting coefficients of x23–x2a with respect to the center points are calculated using x32–x39 as the center points. Where |x 32～38 -x 23～29 |=|x 23～29 -x 32～38 | means that in these two vector operations, there is one data element with a different value and seven data elements with the same value.

[0036] by Figure 5 For example, Figure 5Section 5.1 shows the data used to calculate the weighting coefficients for Vw(4,0; 2,2) and Vw(2,4; 4,2). When calculating Vw(4,0; 2,2), the weighting coefficients of x40–x47 are calculated with x22–x29 as the center points. Similarly, when calculating Vw(2,4; 4,2), the weighting coefficients of x24–x2b are calculated with x42–x49 as the center points. Where |x 42～47 -x 24～29 |=|x 24～29 -x 42～47 | means that in these two vector operations, there are 2 data elements with different values ​​and 6 data elements with the same value. Figure 5 Section 5.2 shows the data used to calculate the weighting coefficients for Vw(4,1;2,2) and Vw(2,3;4,2). When calculating Vw(4,1;2,2), the weighting coefficients of x41–x48 with respect to center points are calculated using x22–x29 as center points. Similarly, when calculating Vw(2,3;4,2), the weighting coefficients of x23–x2a with respect to center points are calculated using x42–x49 as center points. Where |x 42～48 -x 23～29 |=|x 23～29 -x 42～48 | means that in these two vector operations, there is one data element with a different value and seven data elements with the same value.

[0037] As can be seen, when calculating the filter weight coefficients, by utilizing the symmetry of the absolute value of the difference, approximately half of the data can theoretically be recalculated without recalculation. For example... Figure 2 The data elements used for operations in Vw(3,2;2,2) shown are completely symmetrical with those in Vw(2,2;3,2), Vw(4,2;2,2), and Vw(2,2;4,2), and can be calculated only once. However, the completely symmetrical part accounts for a small percentage of the total computation and is not very significant. Figure 3 Vw(2,0;2,2) and Vw(2,4;2,2), Vw(2,1;2,2) and Vw(2,3;2,2), and Figure 4 and Figure 5 For data elements with the same value, only one calculation is needed. However, due to the instruction set architecture of vector processors, it is difficult to efficiently utilize the symmetry of the data. Although most of the data in the above dataset has the same value, there are still some differences, which causes the vector processor to need to perform two calculations, resulting in a large computational load and low processing efficiency.

[0038] This application provides an image processing method to reduce the computational load during image filtering and achieve fast and efficient computation.

[0039] Figure 6 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. The following is in conjunction with... Figure 6 The image processing method provided in the embodiments of this application will be described.

[0040] In S610, image data is acquired.

[0041] The image can be, for example, a digital image. Image data can be, for example, a digital image. Figure 1 The data array in the image can contain a × b data elements. Therefore, image data can be called an a × b data array containing a rows and b columns of data points. The values ​​of a and b can be equal or unequal. For example, the values ​​of a and b can be 4, 22, 16, 32, or 64. The image data used for image filtering can be a portion of the image data; in other words, filtering can be performed on only a portion of the image data. This portion of data can be, for example, located at the edges of the image. Figure 1 For example, the image data can be a 16×16 data array, and the image data used for image filtering can be a portion of the image data, i.e., an 8×12 data array.

[0042] In S620, the first processing unit and the second processing unit are used to calculate the filtering weight coefficients of the image data. The filtering weight coefficients of the image data include the filtering weight coefficients corresponding to the first data element and the filtering weight coefficients corresponding to the second data element. The filtering weight coefficients corresponding to the first data element and the filtering weight coefficients corresponding to the second data element include a first part with the same value and a second part with different values. The first part is calculated by the first processing unit and the second part is calculated by the second processing unit. The first processing unit is a vector processor and the second processing unit is a vector processor or a scalar processor.

[0043] The first processing unit can be used to calculate the vector-level filter weight coefficients; that is, the first processing unit can be a vector processor. Specifically, the first processing unit can perform operations such as absolute difference calculations and maximum value calculations to calculate the weight coefficients of the data relative to the center within the filter size. The first processing unit can also perform operations such as weight coefficient accumulation and summation of weight coefficients with data to calculate the output data for the center point. Figure 3 For example, for the filter weight coefficients Vw(2,0;2,2) and Vw(2,4;2,2), the first vector unit can be used to calculate the filter weight coefficients of Vw(2,0;2,2) and Vw(2,4;2,2) with x24~x29 as the center point and x22~x27 relative to the center point.

[0044] The second processing unit may only perform the absolute value operation of the difference. In some embodiments, the second processing unit may also perform the calculation of filter weight coefficients; that is, the second processing unit may perform operations such as calculating the absolute value of the difference and finding the maximum value to calculate the filter weight coefficients of the data within the filter size relative to the center point. Figure 3 For example, the second processing unit can be used to calculate the weight coefficients of x20 to x21 relative to the center point with x22 to x23 as the center point.

[0045] The second processing unit can be a vector processor, which can perform 2-way or 4-way SIMD operations as needed by the algorithm. The second processing unit can also be a scalar processor, which can execute scalar instructions corresponding to the first processing unit, such as absolute difference instructions and maximum value instructions. The scalar processor can read or / and write data from scalar data registers. The bit width of the scalar data registers can be the same as the single-channel data bit width of the vector data registers in the first processing unit, and the depth of the scalar data registers can be the same as the depth of the vector data registers. For example, 32 32-bit scalar data registers or 32 16-bit SIMD2 scalar data registers can be defined.

[0046] exist Figure 3 In the diagram, Vw(2,0; 2,2) represents the filter weight coefficient corresponding to the first data element Vx22, Vw(2,4; 2,2) represents the filter weight coefficient corresponding to the second data element Vx22, Vw(2,1; 2,2) represents the filter weight coefficient corresponding to the first data element Vx22, and Vw(2,3; 2,2) represents the filter weight coefficient corresponding to the second data element Vx22. Figure 4 In the diagram, Vw(3,0; 2,2) and Vw(2,4; 3,2) are the filter weight coefficients corresponding to the first data element Vx22 and the second data element Vx32, respectively, while Vw(3,1; 2,2) and Vw(2,3; 3,2) are the filter weight coefficients corresponding to the first data element Vx22 and the second data element Vx32, respectively. Figure 5 In the diagram, Vw(4,0;2,2) and Vw(2,4;4,2) are the filter weight coefficients corresponding to the first data element Vx22 and the second data element Vx42, respectively. Vw(4,1;2,2) and Vw(2,3;4,2) are the filter weight coefficients corresponding to the first data element Vx22 and the second data element Vx42, respectively.

[0047] As mentioned earlier, the filter weight coefficients corresponding to the first data element and the second data element include a first part with the same value and a second part with different values. Figure 3In the above, Vw(2,0; 2,2) and Vw(2,4; 2,2) include 6 data elements with the same value and 2 data elements with different values. Vw(2,1; 2,2) and Vw(2,3; 2,2) include 7 data elements with the same value and 1 data element with different value. In other words, when calculating the filter weight coefficients corresponding to the first data element and the second data element, the calculation includes the first part of the operation with the same value and the second part of the operation with the different value. The first part with the same value is calculated by the first processing unit, and the second part with the different value is calculated by the second processing unit. The following is a combination of... Figure 4 Here is an example.

[0048] by Figure 4 Taking section 4.1 as an example, the first processing unit can perform operations on the six elements with the same value in the vector operation. That is, the first processing unit can calculate the filter weight coefficients with x32-37 as the center points and x24-x29 as the corresponding center points. The second processing unit can perform operations on two different elements, that is, calculate the filter weight coefficients with x22-x23 as the center points and x30-x31 as the corresponding center points to obtain Vw(3,0; 2,2), or the second processing unit can execute the filter weight coefficients with x38-x39 as the center points and x2a-x2b as the corresponding center points to obtain Vw(2,4; 3,2). Figure 4 In section 4.2, the first vector unit can perform operations on the same seven data elements; that is, the first processing unit can perform the calculation of the filter weight coefficients of x32 to x38 relative to the center points x23 to x29. The second processing unit can perform operations on one different element. The second processing unit can perform the filter weight coefficients of x31 relative to x22 with the center point x22 to obtain Vw(3,1; 2,2), or the second processing unit can perform the filter weight coefficients of x2a relative to x39 with the center point x39 to obtain Vw(2,3; 3,2). It can be seen that the first processing unit can perform calculations where all values ​​are the same in vector operations, while the second processing unit can perform calculations where all values ​​are different in vector operations. In this way, the same data only needs to be processed once by the first processing unit, avoiding redundant calculations.

[0049] The first processing unit can perform calculations on the first part, including parts with the same values, and it can also perform calculations on parts with different values. Similarly, the second processing unit can perform calculations on the second part, including parts with different values, and it can also perform calculations on the same parts. The following is in conjunction with... Figure 7 An exemplary description will be provided.

[0050] like Figure 7As shown, when calculating Vw(2,0; 2,2), the first processing unit can calculate the filtering weight coefficients with x22~x29 as the center points and x20~x27 as the center points, and the second processing unit can calculate the filtering weight coefficients with x2a~x2b as the center points and x29~x29 as the center points. Thus, the first processing unit outputs the calculation result of Vw(2,0; 2,2), and a portion of the data from the first vector processing result plus the output of the second processing unit equals the calculation result of Vw(2,4; 2,2). When calculating Vw(2,1; 2,2), the first processing unit can calculate the filtering weight coefficients with x22~x29 as the center points and x20~x27 as the center points, and the second processing unit can calculate the filtering weight coefficients with x29~x2a as the center points and x28~x29 as the center points.

[0051] As can be seen, the image processing method provided in this application embodiment can use the first processing unit to perform the same calculation in the filter weight coefficients, so that the part that originally needed to be calculated repeatedly only needs to be calculated once, reducing the amount of calculation and improving the calculation efficiency.

[0052] In some embodiments, linked operation instructions can be used, whereby the first processing unit and the second processing unit perform joint operations according to the linked operation instructions. In some embodiments, the joint operation instructions can be scalar-vector joint operation instructions, that is, the second processing unit can be a scalar processor.

[0053] A joint operation instruction is a single instruction that initiates a joint operation between a first processing unit and a second processing unit. For example, a joint operation instruction may include an instruction for calculating the absolute value of the difference, allowing the first and second processing units to jointly calculate the absolute value of the data difference. A joint operation instruction may also include an instruction for finding the maximum value; that is, a joint operation instruction can enable the first and second processing units to simultaneously calculate the filter coefficients.

[0054] To avoid errors in the instruction encoding process of union operation instructions, the encoding of the registers of the first processing unit can be bound to the encoding of the registers of the second processing unit. That is, the encoding of the registers of the first processing unit is the register number of the second processing unit. For example, the register of the first processing unit is VR, and the register of the second processing unit is DR. The instruction encoding could be, for example, V0:D0, V1:D1, etc. In this way, the encoding of union operation instructions remains similar to the original pure vector version, and the bit fields related to the register encoding do not increase.

[0055] In some embodiments, the first processing unit and the second processing unit may use an aligned loading method to load the data for calculation at once, and use joint operation instructions to complete the corresponding operation at once.

[0056] When the image data used for filtering is an a×b data array, and the filter size is n×n, the data used to calculate the filter weight coefficients is an a+(n-1)×b+(n-1) data array. For example, ... Figure 1 As shown, the image data used for filtering is an 8×12 data array. When the filter size is 5×5, the data used to calculate the filter weight coefficients is a 12×16 data array. When the filter size is 3×3, the data used to calculate the filter weight coefficients is 10×14, and when the filter size is 7×7, the data used to calculate the filter weight coefficients is a 14×18 data array. The following section combines... Figure 7 The image processing method provided in the embodiments of this application will be described.

[0057] Alignment loading can include left alignment or right alignment. The following description uses left alignment as an example. The first processing unit and the second processing unit can load two rows of data for calculating the filter weight coefficients. The data size of both rows is N = N1 + N2, where N1 is the number of data elements in the SIMD instruction of the first processing unit, and N2 is the computational load of the second processing unit.

[0058] like Figure 7 As shown, when calculating Vw(2,0; 2,2), the first processing unit and the second processing unit can load two vectors, each containing 10 data elements, from the left. That is, the first processing unit and the second processing unit can load two rows of data for calculating the filter weight coefficients, one of which is the image data used for filtering. Figure 6As shown, the first processing unit can load x20~x27 and x22~x29, and the second processing unit can load x28~x29 and x2a~x2b. The data volume of these two rows is 10. Eight data elements are loaded and the corresponding operations are performed by the first processing unit, and two data elements are loaded and the corresponding operations are performed by the second processing unit. Both rows are located in a 12×16 data array used to calculate the filter weight coefficients, with one row being a row in an 8×12 data array of image data used for filtering. Using a joint operation instruction, the first vector operation unit performs the calculation of the filter weight coefficients centered on x22~x29 with x20~x27 as the relative center, and the second processing unit performs the calculation of the filter weight coefficients centered on x2a~x2b with x28~x29 as the relative center. In this way, all the data that needs to be calculated is completed without omission or duplication. When calculating Vw(2,1; 2,2), a joint operation instruction is used. The first processing unit performs the calculation of the filter weight coefficients with x21 to x28 as the center and x20 to x27 as the center point. The second processing unit performs the calculation of the filter weight coefficients with x29 to x2a as the center and x28 to x29 as the center point. In this way, one instruction completes the calculation of all data, and the first processing unit can use an aligned loading method to load data, reducing system overhead.

[0059] After the calculation is completed, the operation result of the first processing unit is stored in the register of the first processing unit, and the operation result of the second vector processing unit is stored in the register of the second processing unit. The element shift processing unit can be used to obtain the filtering weight coefficients of the image data based on the above results.

[0060] Continue with Figure 7 For example, when calculating Vw(2,0; 2,2), the first processing unit calculates the filter weight coefficients centered at x22-x29 with x20-x27 as the relative center, and the second processing unit calculates the filter weight coefficients centered at x2a-x2b with x28-x29 as the relative center. The output of the first processing unit is the calculation result of Vw(2,0; 2,2). By shifting the first two bits of the output result of the first processing unit out of its register and shifting the output result of the second processing unit into that register, the calculation result of Vw(2,4; 2,2) can be obtained. When calculating Vw(2,1; 2,2), by shifting the first bit of the output result of the first processing unit out of its register and shifting the first bit of the output result of the second processing unit into that register, the calculation result of Vw(2,1; 2,2) can be obtained. By shifting the first two bits of the output result of the first processing unit out of the register of the first processing unit and shifting the output result of the second processing unit into the register, the calculation result of Vw(2,3;2,2) can be obtained.

[0061] The element shift unit can shift data within registers according to element shift instructions. Instructions for shifting data from the first vector unit register to the second processing unit register or from the second processing unit register to the first processing unit register can be added to the existing element shift processing unit of the vector processor.

[0062] In some embodiments, the computational complexity N2 of the second processing unit is less than the number of data elements N1 in the SIMD instructions of the first processing unit. A smaller second processing unit can be used to achieve the effect of N1+N2, avoiding the need for two or more vector operations. When N1 is sufficiently large, or N2 is much smaller than N1, the second processing unit uses a scalar processing unit. Scalar-vector linkage not only reduces computational complexity but also significantly lowers power consumption. For example, at image edges, extending N1+N2 with scalar vectors is far more convenient and consumes less power than directly extending to N1+N1. It should be understood that this application utilizes the symmetry of the absolute value of the difference, along with scalar-vector linkage instructions, to reduce the number of coefficient operation instructions by half. For other operations with symmetry, such as the square of the difference, the method provided in the embodiments of this application can also be used.

[0063] The image processing method provided in this application, through linked operations, not only saves instruction coding but also reduces computational load and hardware overhead. Taking a filter size of 5×5 as an example, by adding a linked mechanism, this application can reduce the calculation of the original 24 coefficients to 12 when calculating the filter weight coefficients, saving half of the computational load. Furthermore, it saves the instruction overhead during coefficient calculation. Figure 1 For example, before using the method provided in this application, loading two vectors to calculate weight coefficients required 20 data element shifts and 24 vector operations. After using this method, loading two vectors to calculate weight coefficients requires 19 joint element shift instructions, 6 element shift instructions, and 12 vector linkage operation instructions. The reduction in operation instructions, for single-issue instructions, can decrease the number of vector operations. Compared to dual vector processing units, the linkage scheme requires significantly less hardware resources and requires no additional expansion or much additional instruction coding.

[0064] The above text combined Figures 1 to 7 The method embodiments of this application are described in detail below, in conjunction with... Figure 8 The apparatus embodiments of this application are described in detail below. It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments; therefore, any parts not described in detail can be referred to the foregoing method embodiments.

[0065] Figure 8The image data processing apparatus shown may include an acquisition unit 810 and a calculation unit 820. The various units described above will be described exemplarily below.

[0066] Acquisition unit 810 is configured to acquire image data;

[0067] The calculation unit 820 is configured to calculate the filtering weight coefficients of the image data using the first processing unit and the second processing unit. The filtering weight coefficients of the image data include the filtering weight coefficients corresponding to the first data element and the filtering weight coefficients corresponding to the second data element. The filtering weight coefficients corresponding to the first data element and the filtering weight coefficients corresponding to the second data element include a first part with the same value and a second part with different values. The first part is calculated by the first processing unit and the second part is calculated by the second processing unit. The first processing unit is a vector processor and the second processing unit is a vector processor or a scalar processor.

[0068] Optionally, the first processing unit and the second processing unit calculate the first part and the second part according to the linkage operation instruction.

[0069] Optionally, the image data is an a×b data array, and the filter size is n×n. Then, the data used to calculate the filter weight coefficients is an a+(n-1)×b+(n-1) data array. The first vector unit 820 and the second processing unit 830 are further configured to: load two rows of data in the data array used to calculate the filter weight coefficients in an aligned manner, wherein one row of data includes image data, and the amount of data in both rows of data is N=N1+N2, where N1 is the number of data elements in the SIMD instruction of the first processing unit, and N2 is the operation data of the second processing unit; the first processing unit and the second processing unit complete the calculation of the filter weight coefficients of the data in one operation according to the linkage operation instruction.

[0070] Optionally, N2 is less than N1.

[0071] Optionally, the apparatus further includes an element shifting processing unit configured to obtain the filtering weight coefficients of the image data based on the calculation results of the first processing unit and the calculation results of the second processing unit.

[0072] It should be understood that in the embodiments of this application, "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0073] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0074] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0076] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0077] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0078] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can read or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs, DVDs) or semiconductor media (e.g., solid-state disks, SSDs), etc.

[0079] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of processing an image, characterized by, The method comprises: acquiring image data; calculating filter weight coefficients of the image data by using a first processing unit and a second processing unit, wherein the filter weight coefficients of the image data comprise filter weight coefficients corresponding to first data elements and filter weight coefficients corresponding to second data elements, the filter weight coefficients corresponding to the first data elements and the filter weight coefficients corresponding to the second data elements comprise a first part with the same value and a second part with different values, the first part is calculated by the first processing unit, and the second part is calculated by the second processing unit, wherein the first processing unit is a vector processor, and the second processing unit is a vector processor or a scalar processor.

2. The method of claim 1, wherein the first processing unit and the second processing unit calculate the first part and the second part according to a linkage operation instruction.

3. The method of claim 2, wherein, The image data is an a x b data array, the filter size is n x n, and the data used to calculate the filter weight coefficients is an a + (n-1) x b + (n-1) data array, and the method comprises: aligning to load two rows of data in the data array used to calculate the filter weight coefficients, wherein one row of data comprises image data, and the amount of data in the two rows of data is N = N1 + N2, wherein N1 is the number of data elements in a SIMD instruction of the first processing unit, and N2 is operation data of the second processing unit. The first processing unit and the second processing unit complete the calculation of the filter weight coefficients of the data at a time according to the linkage operation instruction.

4. The method of claim 3, wherein, N2 is less than N1.

5. The method of claim 1, wherein, The method further comprises: using an element shift processing unit to obtain the filter weight coefficients of the image data according to the calculation result of the first processing unit and the calculation result of the second processing unit.

6. An apparatus for processing an image, characterized by The apparatus comprises: an acquisition unit configured to acquire image data; a calculation unit configured to calculate filter weight coefficients of the image data by using a first processing unit and a second processing unit, wherein the filter weight coefficients of the image data comprise filter weight coefficients corresponding to first data elements and filter weight coefficients corresponding to second data elements, the filter weight coefficients corresponding to the first data elements and the filter weight coefficients corresponding to the second data elements comprise a first part with the same value and a second part with different values, the first part is calculated by the first processing unit, and the second part is calculated by the second processing unit, wherein the first processing unit is a vector processor, and the second processing unit is a vector processor or a scalar processor.

7. The apparatus of claim 6, wherein, The first processing unit and the second processing unit calculate the first part and the second part according to a linkage operation instruction.

8. The apparatus of claim 7, wherein, The image data is an a x b data array, the filter size is n x n, and the data used to calculate the filter weight coefficients is an a + (n-1) x b + (n-1) data array, and the first processing unit and the second processing unit are further configured to: aligning loading two rows of data in the data array used for calculating the filter weight coefficients, wherein one row of data comprises image data, and the amount of data in the two rows of data is N=N1+N2, wherein N1 is the number of data elements in the first processing unit SIMD instruction, and N2 is the operation data of the second processing unit; the first processing unit and the second processing unit complete the calculation of the filter weight coefficients of the data once according to the linkage operation instruction.

9. The apparatus of claim 8, wherein, N2 is less than N1.

10. The apparatus of claim 6, wherein, The apparatus further comprises: an element shift processing unit configured to obtain the filter weight coefficients of the image data according to the calculation result of the first processing unit and the calculation result of the second processing unit.

Images