A grayscale image scaling optimization method of OpenCV logic

Abstract

The application provides a gray-scale image scaling optimization method of OpenCV logic, uses a simd instruction to obtain corresponding four groups of input image data from an input image, four data in each group, and then parallelly calculates the four groups of data through a SIMD instruction to obtain corresponding output data and store the output data in an output image address. Thus, the parallel operation can greatly accelerate the resize time, thereby meeting the needs of the industry.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a grayscale image scaling optimization method based on OpenCV logic. Background Technology

[0002] In current technologies, OpenCV's bilinear interpolation logic is commonly used as an image interpolation method, and is widely applied in signal processing, digital image and video processing. Bilinear interpolation is an extension of linear interpolation with two variable functions; its core idea is to perform linear interpolation in two directions separately.

[0003] However, the existing technology has the following drawbacks:

[0004] 1. The bilinear interpolation logic implemented using OpenCV cannot be used on hardware platforms (chip platforms) and runs slowly, making it unsuitable for industrial needs.

[0005] 2. The bilinear interpolation logic in C language can only calculate the data of a single pixel at a time, which is very slow and completely unsuitable for industrial needs.

[0006] In addition, the technical terms in this field include:

[0007] 1. SIMD128: Single Instruction Multiple Data, meaning that a single instruction can execute multiple data streams, thus improving program processing speed. (Each register is 128 bits wide). Summary of the Invention

[0008] To address the aforementioned issues, the purpose of this application is to optimize OpenCV's C language implementation, significantly reducing processing time and enabling its industrial application.

[0009] Specifically, this invention provides a grayscale image scaling optimization method based on OpenCV logic. The method uses SIMD instructions to obtain four sets of input image data from the input image, each set containing four data points. Then, it uses SIMD instructions to perform parallel computation on the four sets of data to obtain the corresponding output data and stores it in the output image address. The method includes:

[0010] S1. Using SIMD instructions, read four groups of 128-bit input image data in parallel from the input image address, each group containing four pixels, and load the input-output mapping weight data into a register; including:

[0011] The instruction v16u8 pixel11=_mx128_lu1q(&ptr_pixel_11[0],0) is used to read 128 bits of data from the address ptr_pixel_11 into the pixel11 register;

[0012] Repeat the above operation three times, reading data from the addresses ptr_pixel_12, ptr_pixel_21, and ptr_pixel_22 into the corresponding registers respectively;

[0013] The instruction `v16u8 idx_0 = _mx128_lu1q(&w_pixel_0[dst_x],0)` reads the mapping data from address `w_pixel_0` into register `idx_0`.

[0014] S2. Use SIMD instructions for data rearrangement and bit-width conversion; including:

[0015] S2.1, using the instruction v16i8 src_w_0=_mx128_shufv(pixel11,pixel11,idx_0) to select data from the pixel11 register and transfer it to the src_w_0 register based on idx_0;

[0016] Repeat the above operation three times to obtain the data from the src_w_1, src_h_0, and src_h_1 registers respectively; S2.2, use the instruction v4u32 in_w_0=_mx128_shufv(zero,src_w_0,b_to_w) to convert the first four 8-bit data of src_w_0 into 32-bit data in the in_w_0 register;

[0017] Repeat the above operation three times to obtain the data in the in_w_1, in_h_0, and in_h_1 registers respectively;

[0018] S3. Weighted calculations using SIMD instructions; including:

[0019] S3.1, use the instruction in_w_0=_mx128_mul_w(in_w_0,w0lambda_1) to multiply in_w_0 with the weight register w0lambda_1;

[0020] Repeat the above operation three times to calculate the product of in_w_vr1, in_h_vr0, and in_h_vr2 with the corresponding weights respectively;

[0021] S3.2, use the instruction in_w_0=_mx128_add_w(in_w_0,in_w_1) to add in_w_0 and in_w_1;

[0022] Repeat the above operation to add in_h_0 and in_h_1;

[0023] S4. Perform displacement and weighted calculations using SIMD instructions; including:

[0024] S4.1, use the instruction in_w_0=_mx128_srli_w(in_w_0,4) to shift the data in the in_w_0 register to the right by 4 bits and store the data back into the in_w_0 register;

[0025] S4.2, use the instruction in_h_0=_mx128_srli_w(in_h_0,4) to shift the data in the in_h_0 register to the right by 4 bits and store the data back into the in_h_0 register;

[0026] S4.3, use the instruction in_w_0=_mx128_mul_w(in_w_0,h0lambda_1) to multiply in_w_0 with the h-dimensional weight h0lambda_1; use the instruction in_h_0=_mx128_mul_w(in_h_0,h1lambda_1) to multiply in_h_0 with the h-dimensional weight h1lambda_1;

[0027] S4.4, using in_h_0=_mx128_srli_w(in_h_0,16) and in_w_0=

[0028] The instruction _mx128_srli_w(in_w_0,4) performs a shift operation;

[0029] S5. Use SIMD instructions to perform result calculations and storage; including:

[0030] S5.1, use the instruction vr_out = _mx128_add_w(in_h_0,in_w_0) to add in_h_0 and in_w_0; use the instruction vr_out = _mx128_add_w(vr_out,bias) to add the bias value; S5.2, use the instruction vr_out = _mx128_srli_w(vr_out,2) to right shift by 2 bits; use the instruction v16u8out_value = _mx128_shufv(vr_out,vr_out,w_to_b) to convert the 32-bit data to 8 bits;

[0031] S5.3 uses the instruction _mx128_su1q(out_value,dst_addr,0) to store the result at the output address.

[0032] In step S1, the instruction v16u8 pixel11 = _mx128_lu1q(&ptr_pixel_11[0],0) is used to read 128 bits of data from the address ptr_pixel_11. The ptr_pixel_11 is the address calculated from the first input image coordinates mapped from dst. v16u8 pixel11 = _mx128_lu1q(&ptr_pixel_11[0],0) is a simd128 instruction, which means to read 128 bits of data from the address ptr_pixel_11 and store the data in the pixel11 register. v16u8 represents 16 unsigned 8-bit data, which is exactly 128 bits of data.

[0033] By repeating the above operation three times, the data registers ptr_pixel_12, ptr_pixel_21, and ptr_pixel_22 in the addresses calculated from the coordinates of the other three input image points can be obtained.

[0034] Then, v16u8 idx_0 = _mx128_lu1q(&w_pixel_0[dst_x],0) is used to read the input and output mapping from the address w_pixel_0.

[0035] The mapping relationship between the input and output is obtained by the following calculation formula. The input-output mapping method based on loop calculation includes the following steps:

[0036] S1.1. Iterate through the blocks in the width direction of the target image:

[0037] The target image width valid_dst_w is divided into blocks according to width_cycs, and the number of loops cycs_i is calculated.

[0038] S1.2. Traverse the pixel indices within the current block:

[0039] Calculate the current target pixel index dst_x = cycs_i * width_cycs + i;

[0040] S1.3. Calculate the source image coordinates and weights:

[0041] Based on the scaling factor scale_x, calculate the floating-point source coordinates src_fx_round corresponding to the current target pixel dst_x: src_fx_round = (dst_x + 0.5) * scale_x - 0.5;

[0042] Rounding up to the integer source coordinates src_x = (int)src_fx_round, and constraining them to not exceed the source image width box_w-1;

[0043] Calculate the boundary marker bias. If src_x does not reach the right boundary, then bias = 1; otherwise, bias = 0. S1.4. Calculate the bilinear interpolation weights:

[0044] Calculate the horizontal weights w_x1 = (src_fx_round - src_x) * 2048 and w_x0 = (1 - src_fx_round + src_x) * 2048, and store them in w1lambda[dst_x] and w0lambda[dst_x];

[0045] S1.5. Calculate the data rearrangement index:

[0046] Calculate the starting source coordinates of the current block: start_idx = ((cycs_i * width_cycs + 0.5) * scale_x - 0.5);

[0047] Calculate and store w_pixel_0[dst_x] and w_pixel_1[dst_x] for data selection in subsequent SIMD instructions, where:

[0048] w_pixel_0[dst_x]=(src_x-start_idx)*2>0? (src_x-start_idx)*2:0; w_pixel_1[dst_x]=(src_x-start_idx+bias)*2.

[0049] In step S1, due to the limitation of the simd128 register, only 4 data can be processed at a time. The weight is 32 bits, 32*4=128 bits, so width_cycs is equal to 4. That is, width_cycs is selected as 4 to match the requirement of the 128-bit register of the SIMD instruction to process 4 32-bit weight data in parallel.

[0050] The weights w_x1 and w_x0 are represented in fixed-point numbers, and the floating-point weights are converted to 11-bit precision integers by multiplying by 2048;

[0051] The index values ​​stored in w_pixel_0 and w_pixel_1 are used for the data rearrangement operation of the SIMD instruction _mx128_shufv to select the corresponding pixel data from the input register.

[0052] In step S2, the data rearrangement operation selects data from a specific position in the input register through the relational register idx_0, where the idx_0 register contains 16 index values, each index value corresponding to an 8-bit data position in the input register; further including:

[0053] S2.1, using v16i8 src_w_0 = _mx128_shufv(pixel11,pixel11,idx_0), the data is read from the pixel11 register according to the idx_0 relational register and stored in the src_w_0 register. Here, _mx128_shufv(pixel11,pixel11,idx_0) is a SIMD instruction, which reads data from the pixel11 and pixel11 registers through the idx_0 relational register and stores the data in the src_w_0 register.

[0054] Repeat the above operation three more times to obtain the required input data registers src_w_1, src_h_0, and src_h_1.

[0055] S2.2, then the instruction v4u32 in_w_0=_mx128_shufv(zero,src_w_0,b_to_w) is used to expand the first four 8-bit data of the src_w_0 register into 32-bit data and store the data in the in_w_0 register. The zero register is filled with all 0s, and the data in the b_to_w register is {0,1,1,1,2,3,3,3,4,5,5,5,6,7,7,7}.

[0056] Repeat the above operation three times, reading the first four 8-bit data from the src_w_1, src_h_0, and src_h_1 registers, and converting the data into 32-bit data and storing it in the in_w_1, in_w_0, and in_w_1 registers.

[0057] In step S3, the weighted calculation of four pixel data is processed in parallel, where the weights are 32-bit fixed-point numbers, and the four weights occupy a total of 128 bits of registers; further including:

[0058] S3.1, using in_w_0=_mx128_mul_w(in_w_0,w0lambda_1), the instruction in_w_0=_mx128_mul_w(in_w_0,w0lambda_1) belongs to the SIMD instruction, which multiplies the data in the in_w_0 register and the w0lambda_1 register bitwise, and puts the result back into the in_w_0 register. w0lambda_1 is the bilinear interpolation w-direction weight, which is calculated in advance and written into the register. The above operation is repeated three times to obtain the data of the in_w_vr1, in_h_vr0, in_h_vr2 registers and the weight registers w1_lamdba_1, w0_lambda_1, w1_lambda_1 registers, and the multiplied data is stored back into the corresponding registers.

[0059] S3.2, followed by in_w_0 = _mx128_add_w(in_w_0,in_w_1); the instruction in_w_0 = _mx128_add_w(in_w_0,in_w_1) belongs to the SIMD instruction, which adds the data in registers in_w_0 and in_w_1 bitwise, and stores the added data back in register in_w_0. The above operation is repeated to add the data in registers in_h_0 and in_h_1 and store the data back in register in_h_0.

[0060] Step S4 further includes:

[0061] S4.1, using in_w_0 = _mx128_srli_w(in_w_0, 4); instruction in_w_0 =

[0062] _mx128_srli_w(in_w_0,4) is a SIMD instruction that shifts the data in the in_w_0 register four bits to the right and saves the data back to the in_w_0 register.

[0063] S4.2, use in_h_0 = _mx128_srli_w(in_h_0, 4); shift the data in the in_h_0 register right by 4 bits and save the data back to the in_h_0 register;

[0064] S4.3, followed by using in_w_0 = _mx128_mul_w(in_w_0, h0lambda_1) and in_h_0 = _mx128_mul_w(in_h_0, h1lambda_1); this instruction multiplies in_w_0 and in_h_0 by the corresponding h-dimensional weight registers h0lambda_1 and h1lambda_1 data respectively, and saves the data back to the in_w_0 register and the inh_0 register;

[0065] S4.4, then use the instructions in_h_0 = _mx128_srli_w(in_h_0,16) and in_w_0 =

[0066] The instruction _mx128_srli_w(in_w_0,4) shifts the data in the in_h_0 and in_w_0 registers 16 bits to the right.

[0067] The result calculation in step S5 includes the superposition of interpolation results in the horizontal and vertical directions, and rounding with a fixed offset value of 2; further including:

[0068] S5.1, the instructions vr_out = _mx128_add_w(in_h_0,in_w_0) and vr_out = _mx128_add_w(vr_out,bias) are used to add the data in the in_h_0 register to the data in the in_w_0 register and to the data in the bias register, and then store the data in the vr_out register; where the bias register always contains the number 2.

[0069] S5.2, then use the instructions vr_out = _mx128_srli_w(vr_out,2) and v16u8 out_value = _mx128_shufv(vr_out,vr_out,w_to_b) to shift the data in the vr_out register two bits to the right and convert the data from 32 bits to 8 bits, and store the data in the out_value register;

[0070] S5.3 Finally, _mx128_su1q(out_value,dst_addr,0) is used to store the data in the out_value register into the output address dst_addr.

[0071] Therefore, the grayscale image scaling optimization method based on SimD and OpenCV logic proposed in this scheme has the following advantages:

[0072] 1. Using the simd instruction to calculate 4 pixels in parallel, while C language can only calculate a single pixel at a time, the current time efficiency is optimized by 69% compared to the C language implementation.

[0073] 2. It is perfectly compatible with Junzheng T32 / T33 chips. Attached Figure Description

[0074] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0075] Figure 1 This is a schematic diagram of the C language logic code for converting an existing RGBA888 image to an RGB565 image.

[0076] Figure 2 This is a flowchart illustrating the method described in this application.

[0077] Figure 3 This is a code diagram illustrating how the mapping relationship between inputs and outputs in method step S1 of this application is obtained using the following calculation formula.

[0078] Figure 4 This is a schematic diagram illustrating the implementation of method step S2 in this application. Detailed Implementation

[0079] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0080] The existing C language logic for OpenCV's bilinear interpolation of grayscale images is as follows: Figure 1 As shown, it includes:

[0081] 1. Calculate the scaling ratio: Based on the source image dimensions (src_w, src_h) and the target image dimensions (dst_w, dst_h), calculate the horizontal scaling ratio scale_x and the vertical scaling ratio scale_y; 2. Traverse the target image pixels: For each pixel position (dy, dx) of the target image, perform the following operations:

[0082] 2.1 Calculate the floating-point coordinates of the source image: Based on the current target pixel position and scaling ratio, calculate the corresponding floating-point coordinates (fy, fx) of the source image;

[0083] 2.2 Separating Integer and Decimal Parts: Decompose the floating-point coordinates into an integer part (sy, sx) and a decimal part (fy, fx) for interpolation calculations;

[0084] 2.3 Boundary checking and correction: If the integer coordinates are less than 0, set them to 0; if they are greater than or equal to the source image size minus 1, set them to the maximum valid coordinates.

[0085] 2.4 Calculate interpolation weights: Based on the fractional part, calculate the horizontal interpolation weights (alpha0, alpha1) and the vertical interpolation weights (beta0, beta1), and optimize using a fixed-point ratio of 2048. The interpolation weight calculation uses saturate casting to ensure the validity of the numerical range. Obtain neighboring pixel values: Read the four pixels (p00, p01, p10, p11) adjacent to (sy, sx) from the source image.

[0086] 2.5 Perform bilinear interpolation: First, perform horizontal interpolation to calculate the intermediate value (val0, val1) and shift it 4 bits to the right. Then, perform vertical interpolation to calculate the intermediate value and shift it 16 bits to the right. Finally, add 2 to the result and shift it 2 bits to the right to round it to obtain the final pixel value val.

[0087] 2.6 Storing the result: Store the final pixel value val in the (dy,dx) position of the target image.

[0088] The main logic of OpenCV's grayscale image resizing is as shown above: it maps the coordinates of the four input images back from the output image dst, then obtains the four pixel values ​​from the input image src according to the input image coordinates, performs bilinear interpolation to obtain a single output pixel, and saves the data into the address of the output image dst.

[0089] This invention provides a grayscale image scaling optimization method based on SimD OpenCV logic. The main optimization logic involves using SimD instructions to obtain four sets of input image data (four data points per set) from the input image, then using SimD instructions to perform parallel computation on these four sets of data to obtain the corresponding output data and store it in the output image address. This parallel operation significantly accelerates the resizing time, thus meeting industrial requirements. The specific operation is as follows: Figure 2 As shown:

[0090] Step S1. Use the instruction v16u8 pixel11=_mx128_lu1q(&ptr_pixel_11[0],0) (v16i8pixel11=_mx128_lu1q(&ptr_pixel_11[0],0) is a simd128 instruction, which means to read 128 bits of data from the address ptr_pixel_11 and store the data in the pixel11 register. v16u8 represents 16 unsigned 8-bit data, which is exactly 128 bits of data), to read 128 bits of data from the address ptr_pixel_11 (prt_pixel_11 is the address calculated from the first input image coordinates mapped from dst);

[0091] By repeating the above operation three times, the data registers ptr_pixel_12, ptr_pixel_21, and ptr_pixel_22 in the addresses calculated from the coordinates of the other three input image points can be obtained.

[0092] Then, v16u8 idx_0 = _mx128_lu1q(&w_pixel_0[dst_x],0) is used to read the input and output mapping from the address w_pixel_0.

[0093] The mapping relationship between input and output is obtained by the following formula, as shown in the code example. Figure 3 As shown: The input-output mapping method based on loop computation includes the following steps:

[0094] S1.1. Iterate through the blocks in the width direction of the target image:

[0095] The target image width valid_dst_w is divided into blocks according to width_cycs (preferably 4), and the number of loops cycs_i is calculated;

[0096] S1.2. Traverse the pixel indices within the current block:

[0097] Calculate the current target pixel index dst_x = cycs_i * width_cycs + i;

[0098] S1.3. Calculate the source image coordinates and weights:

[0099] Based on the scaling factor scale_x, calculate the floating-point source coordinates src_fx_round corresponding to the current target pixel dst_x: src_fx_round = (dst_x + 0.5) * scale_x - 0.5;

[0100] Rounding up to the integer source coordinates src_x = (int)src_fx_round, and constraining them to not exceed the source image width box_w-1;

[0101] Calculate the boundary marker bias. If src_x does not reach the right boundary, then bias = 1; otherwise, bias = 0. S1.4. Calculate the bilinear interpolation weights:

[0102] Calculate the horizontal weights w_x1 = (src_fx_round - src_x) * 2048 and w_x0 = (1 - src_fx_round + src_x) * 2048, and store them in w1lambda[dst_x] and w0lambda[dst_x];

[0103] S1.5. Calculate the data rearrangement index:

[0104] Calculate the starting source coordinates of the current block: start_idx = ((cycs_i * width_cycs + 0.5) * scale_x - 0.5);

[0105] Calculate and store w_pixel_0[dst_x] and w_pixel_1[dst_x] for data selection in subsequent SIMD instructions, where:

[0106] w_pixel_0[dst_x]=(src_x-start_idx)*2>0? (src_x-start_idx)*2:0;

[0107] w_pixel_1[dst_x]=(src_x-start_idx+bias)*2.

[0108] Due to the limitation of the simd128 register, only 4 data points can be processed at a time (the weights are 32 bits, 32 * 4 = 128), so width_cycs equals 4. That is, the width_cycs value is 4 to match the requirement of the SIMD instruction's 128-bit register to process 4 32-bit weight data points in parallel.

[0109] The weights w_x1 and w_x0 are represented as fixed-point numbers, and the floating-point weights are converted to 11-bit precision integers by multiplying by 2048.

[0110] The index values ​​stored in w_pixel_0 and w_pixel_1 are used for the data rearrangement operation of the SIMD instruction _mx128_shufv to select the corresponding pixel data from the input register.

[0111] Step S2. Use v16i8 src_w_0 = _mx128_shufv(pixel11,pixel11,idx_0); (This is a SIMD instruction, which reads data from pixel11 and pixel11 registers through the relational register idx_0 and stores the data in the src_w_0 register. A detailed explanation is provided later.) Read data from the pixel11 register according to the relational register idx_0 and store the data in src_w_0. Repeat the above operation three times to obtain the required input data registers src_w_1, src_h_0, and src_h_1. Next, the instruction `v4u32 in_w_0=_mx128_shufv(zero,src_w_0,b_to_w)` is used; (the zero register contains only 0s, and the b_to_w register contains {0,1,1,1,2,3,3,3,4,5,5,5,6,7,7,7});) expands the first four 8-bit data in the src_w_0 register into 32-bit data and stores the data in the in_w_0 register;

[0112] Repeat the above operation three times, reading the first four 8-bit data from the src_w_1, src_h_0, and src_h_1 registers, and converting the data into 32-bit data and storing it in the in_w_1, in_w_0, and in_w_1 registers.

[0113] like Figure 4 As shown in the diagram, the data in the left-hand pixel0 register are numbered 1, 3, 5, 7, ..., 31 (a normal register can store 16 8-bit data, hence the numbers 1-32. However, due to space limitations, ...). Figure 4Only four data points are shown. In this example, pixel0 register stores data from 1 to 16, and pixel1 register stores data from 17 to 32. The codes for each data point in the right-hand pixel1 register are 0, 2, 4, 6, ..., 30. The data in the relational register idx_0 is the code. We retrieve the corresponding code data from the pixel0 and pixel1 registers. For example, if idx_0[0] = 0, then the data with code 0 will be stored in the src_w[0] register. Therefore, when the relational register data is 1, 9, 17, 25, 0, 8, 16, 24, data with codes 1, 9, 17, 25 will be selected from the pixel0 register and data with codes 0, 8, 16, 24 will be selected from the pixel1 register, and the data will be stored in the src_w register.

[0114] Step S3. Use the instruction `in_w_0 = _mx128_mul_w(in_w_0, w0lambda_1);` (which is a SIMD instruction; it multiplies the data in the `in_w_0` register and the `w0lambda_1` register bitwise, and puts the result back into the `in_w_0` register. `W0lambda_1` is the bilinear interpolation weight in the w-direction, which we calculate and write into the register beforehand). Repeat the above operation three more times to obtain the data from the multiplication of the `in_w_vr1`, `in_h_vr0`, and `in_h_vr2` registers with the corresponding weight registers `w1_lamdba_1`, `w0_lambda_1`, and `w1_lambda_1` registers, and store the multiplied data back into the corresponding registers. Next, the instruction `in_w_0 = _mx128_add_w(in_w_0,in_w_1);` is used. (`in_w_0 = _mx128_add_w(in_w_0,in_w_1);` is a SIMD instruction that adds the data in registers `in_w_0` and `in_w_1` bitwise and stores the sum back into register `in_w_0`. This process is repeated to add the data in registers `in_h_0` and `in_h_1` and store the sum back into register `in_h_0`.

[0115] Step S4. Use the instruction in_w_0 = _mx128_srli_w(in_w_0,4); (in_w_0 = _mx128_srli_w(in_w_0,4); belongs to SIMD instructions, which shifts the data in the in_w_0 register four bits to the right and saves the data back to the in_w_0 register) and in_h_0 = _mx128_srli_w(in_h_0,4); to shift the data in the in_h_0 register four bits to the right and save the data back to the in_h_0 register. Next, the instructions `in_w_0 = _mx128_mul_w(in_w_0, h0lambda_1);` and `in_h_0 = _mx128_mul_w(in_h_0, h1lambda_1);` multiply `in_w_0` and `in_h_0` by the corresponding weight registers `h0lambda_1` and `h1lambda_1` for the h dimension, respectively, and save the data back to the `in_w_0` and `in_h_0` registers. Then, the instructions `in_h_0 = _mx128_srli_w(in_h_0, 16);` and `in_w_0 = _mx128_srli_w(in_w_0, 4);` shift the data in the `in_h_0` and `in_w_0` registers right by 16 bits.

[0116] Step S5. Use the instructions `vr_out = _mx128_add_w(in_h_0,in_w_0);` and `vr_out = _mx128_add_w(vr_out,bias)` (where `bias` contains the value 2) to add the data in registers `in_h_0` and `in_w_0`, as well as the data in register `bias`, and store the result in register `vr_out`. Then use the instructions `vr_out = _mx128_srli_w(vr_out,2);` and `v16u8 out_value = _mx128_shufv(vr_out,vr_out,w_to_b);` to right-shift the data in register `vr_out` by two bits and convert the data from 32 bits to 8 bits, and store the result in register `out_value`. Finally, _mx128_su1q(out_value,dst_addr,0) is used to store the data in the out_value register into the output address dst_addr.

[0117] In summary, using the SIMD optimization method in steps S1-S5 above, four 8-bit data can be processed in parallel simultaneously, greatly reducing the time of OpenCV's grayscale image scaling process (by 69%).

[0118] In summary, this invention proposes a novel grayscale image scaling optimization method based on SIMD OpenCV logic; compared with the latest methods in the current technology, it also achieves a significant improvement in time efficiency.

[0119] This technical solution can be applied to the Junzheng T32 / T33 series chips and similar chips.

[0120] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A grayscale image scaling optimization method based on OpenCV logic, characterized in that, The method uses SIMD instructions to obtain four sets of input image data from the input image, with four data points in each set. Then, it uses SIMD instructions to perform parallel computation on the four sets of data to obtain the corresponding output data, which is then stored in the output image address. This includes: S1. Using SIMD instructions, read four groups of 128-bit input image data in parallel from the input image address, each group containing four pixels, and load the input-output mapping weight data into a register; including: The instruction v16u8 pixel11=_mx128_lu1q(&ptr_pixel_11[0],0) is used to read 128 bits of data from the address ptr_pixel_11 into the pixel11 register; Repeat the above operation three times, reading data from the addresses ptr_pixel_12, ptr_pixel_21, and ptr_pixel_22 into the corresponding registers respectively; The instruction `v16u8 idx_0 = _mx128_lu1q(&w_pixel_0[dst_x],0)` reads the mapping data from address `w_pixel_0` into register `idx_0`. S2. Use SIMD instructions for data rearrangement and bit-width conversion; including: S2.1, using the instruction v16i8 src_w_0=_mx128_shufv(pixel11,pixel11,idx_0) to select data from the pixel11 register and transfer it to the src_w_0 register based on idx_0; Repeat the above operation three times to obtain the data from the src_w_1, src_h_0, and src_h_1 registers respectively; S2.2, use the instruction v4u32 in_w_0=_mx128_shufv(zero,src_w_0,b_to_w) to convert the first 4 8-bit data of src_w_0 into 32-bit data into the in_w_0 register; Repeat the above operation three times to obtain the data in the in_w_1, in_h_0, and in_h_1 registers respectively; S3. Weighted calculations using SIMD instructions; including: S3.1, use the instruction in_w_0=_mx128_mul_w(in_w_0,w0lambda_1) to multiply in_w_0 with the weight register w0lambda_1; Repeat the above operation three times to calculate the product of in_w_vr1, in_h_vr0, and in_h_vr2 with the corresponding weights respectively; S3.2, use the instruction in_w_0=_mx128_add_w(in_w_0,in_w_1) to add in_w_0 and in_w_1; Repeat the above operation to add in_h_0 and in_h_1; S4. Perform displacement and weighted calculations using SIMD instructions; including: S4.1, use the instruction in_w_0=_mx128_srli_w(in_w_0,4) to shift the data in the in_w_0 register to the right by 4 bits and store the data back into the in_w_0 register; S4.2, use the instruction in_h_0=_mx128_srli_w(in_h_0,4) to shift the data in the in_h_0 register to the right by 4 bits and store the data back into the in_h_0 register; S4.3, use the instruction in_w_0=_mx128_mul_w(in_w_0,h0lambda_1) to multiply in_w_0 with the h-dimensional weight h0lambda_1; use the instruction in_h_0=_mx128_mul_w(in_h_0,h1lambda_1) to multiply in_h_0 with the h-dimensional weight h1lambda_1; S4.4, use the instructions in_h_0=_mx128_srli_w(in_h_0,16) and in_w_0=_mx128_srli_w(in_w_0,4) to perform the shift operation; S5. Use SIMD instructions to perform result calculations and storage; including: S5.1, use the instruction vr_out = _mx128_add_w(in_h_0,in_w_0) to add in_h_0 and in_w_0; use the instruction vr_out = _mx128_add_w(vr_out,bias) to add the bias value; S5.2, use the instruction vr_out = _mx128_srli_w(vr_out,2) to right shift by 2 bits; use the instruction v16u8out_value = _mx128_shufv(vr_out,vr_out,w_to_b) to convert the 32-bit data to 8 bits; S5.3 uses the instruction _mx128_su1q(out_value,dst_addr,0) to store the result at the output address.

2. The method for optimizing grayscale image scaling in OpenCV logic according to claim 1, characterized in that, In step S1, the instruction v16u8 pixel11 = _mx128_lu1q(&ptr_pixel_11[0],0) is used to read 128 bits of data from the address ptr_pixel_11. The ptr_pixel_11 is the address calculated from the first input image coordinates mapped from dst. v16u8 pixel11 = _mx128_lu1q(&ptr_pixel_11[0],0) is a simd128 instruction, which means to read 128 bits of data from the address ptr_pixel_11 and store the data in the pixel11 register. v16u8 represents 16 unsigned 8-bit data, which is exactly 128 bits of data. By repeating the above operation three times, the data registers ptr_pixel_12, ptr_pixel_21, and ptr_pixel_22 in the addresses calculated from the coordinates of the other three input image points can be obtained. Then, v16u8 idx_0 = _mx128_lu1q(&w_pixel_0[dst_x],0) is used to read the input and output mapping from the address w_pixel_0.

3. The method for optimizing grayscale image scaling in OpenCV logic according to claim 2, characterized in that, The mapping relationship between the input and output is obtained by the following calculation formula. The input-output mapping method based on loop calculation includes the following steps: S1.

1. Iterate through the blocks in the width direction of the target image: The target image width valid_dst_w is divided into blocks according to width_cycs, and the number of loops cycs_i is calculated. S1.

2. Traverse the pixel indices within the current block: Calculate the current target pixel index dst_x = cycs_i * width_cycs + i; S1.

3. Calculate the source image coordinates and weights: Based on the scaling factor scale_x, calculate the floating-point source coordinates src_fx_round corresponding to the current target pixel dst_x: src_fx_round = (dst_x + 0.5) * scale_x - 0.5; Rounding up to the integer source coordinates src_x = (int)src_fx_round, and constraining them to not exceed the source image width box_w-1; Calculate the boundary marker bias. If src_x does not reach the right boundary, then bias = 1; otherwise, bias = 0. S1.

4. Calculate the bilinear interpolation weights: Calculate the horizontal weights w_x1 = (src_fx_round - src_x) * 2048 and w_x0 = (1 - src_fx_round + src_x) * 2048, and store them in w1lambda[dst_x] and w0lambda[dst_x]; S1.

5. Calculate the data rearrangement index: Calculate the starting source coordinates of the current block: start_idx = ((cycs_i * width_cycs + 0.5) * scale_x - 0.5); Calculate and store w_pixel_0[dst_x] and w_pixel_1[dst_x] for data selection in subsequent SIMD instructions, where: w_pixel_0[dst_x]=(src_x-start_idx)*2>0? (src_x-start_idx)*2:0; w_pixel_1[dst_x]=(src_x-start_idx+bias)*2.

4. The method for optimizing grayscale image scaling in OpenCV logic according to claim 3, characterized in that, In step S1, due to the limitation of the simd128 register, only 4 data can be processed at a time. The weight is 32 bits, 32*4=128 bits, so width_cycs is equal to 4. That is, width_cycs is selected as 4 to match the requirement of the 128-bit register of the SIMD instruction to process 4 32-bit weight data in parallel. The weights w_x1 and w_x0 are represented in fixed-point numbers, and the floating-point weights are converted to 11-bit precision integers by multiplying by 2048; The index values ​​stored in w_pixel_0 and w_pixel_1 are used for the data rearrangement operation of the SIMD instruction _mx128_shufv to select the corresponding pixel data from the input register.

5. The method for optimizing grayscale image scaling in OpenCV logic according to claim 1, characterized in that, In step S2, the data rearrangement operation selects data from a specific position in the input register through the relational register idx_0, where the idx_0 register contains 16 index values, each index value corresponding to an 8-bit data position in the input register; further including: S2.1, using v16i8 src_w_0 = _mx128_shufv(pixel11,pixel11,idx_0), the data is read from the pixel11 register according to the idx_0 relational register and stored in the src_w_0 register. Here, _mx128_shufv(pixel11,pixel11,idx_0) is a SIMD instruction, which reads data from the pixel11 and pixel11 registers through the idx_0 relational register and stores the data in the src_w_0 register. Repeat the above operation three more times to obtain the required input data registers src_w_1, src_h_0, and src_h_1. S2.2, then the instruction v4u32 in_w_0=_mx128_shufv(zero,src_w_0,b_to_w) is used to expand the first four 8-bit data of the src_w_0 register into 32-bit data and store the data in the in_w_0 register. The zero register is filled with all 0s, and the data in the b_to_w register is {0,1,1,1,2,3,3,3,4,5,5,5,6,7,7,7}. Repeat the above operation three times, reading the first four 8-bit data from the src_w_1, src_h_0, and src_h_1 registers, and converting the data into 32-bit data and storing it in the in_w_1, in_w_0, and in_w_1 registers.

6. The method for optimizing grayscale image scaling in OpenCV logic according to claim 1, characterized in that, In step S3, the weighted calculation of four pixel data is processed in parallel, where the weights are 32-bit fixed-point numbers, and the four weights occupy a total of 128 bits of registers; further including: S3.1, using in_w_0=_mx128_mul_w(in_w_0,w0lambda_1), the instruction in_w_0=_mx128_mul_w(in_w_0,w0lambda_1) belongs to the SIMD instruction, which multiplies the data in the in_w_0 register and the w0lambda_1 register bitwise, and puts the result back into the in_w_0 register. w0lambda_1 is the bilinear interpolation w-direction weight, which is calculated in advance and written into the register. The above operation is repeated three times to obtain the data of the in_w_vr1, in_h_vr0, in_h_vr2 registers and the weight registers w1_lamdba_1, w0_lambda_1, w1_lambda_1 registers, and the multiplied data is stored back into the corresponding registers. S3.2, followed by the use of in_w_0 = _mx128_add_w(in_w_0,in_w_1); the instruction in_w_0 = _mx128_add_w(in_w_0,in_w_1) belongs to the SIMD instruction, which adds the data in registers in_w_0 and in_w_1 bitwise, and stores the added data back into register in_w_0. The above operation is repeated to add the data in registers in_h_0 and in_h_1, and store the data back into register in_h_0.

7. The method for optimizing grayscale image scaling in OpenCV logic according to claim 1, characterized in that, Step S4 further includes: S4.1, use in_w_0=_mx128_srli_w(in_w_0,4); the instruction in_w_0=_mx128_srli_w(in_w_0,4) belongs to the SIMD instruction, which shifts the data in the in_w_0 register four bits to the right and saves the data back to the in_w_0 register; S4.2, using in_h_0 = _mx128_srli_w(in_h_0, 4); shifts the data in the in_h_0 register four bits to the right and stores the data back into the in_h_0 register; S4.3, followed by using in_w_0 = _mx128_mul_w(in_w_0, h0lambda_1) and in_h_0 = _mx128_mul_w(in_h_0, h1lambda_1); this instruction multiplies in_w_0 and in_h_0 by the corresponding h-dimensional weight registers h0lambda_1 and h1lambda_1 data respectively, and saves the data back to the in_w_0 register and the inh_0 register; S4.4 Then use the instructions in_h_0 = _mx128_srli_w(in_h_0,16) and in_w_0 = _mx128_srli_w(in_w_0,4) to right shift the data in the in_h_0 and in_w_0 registers by 16 bits.

8. The method for optimizing grayscale image scaling in OpenCV logic according to claim 1, characterized in that, The result calculation in step S5 includes the superposition of interpolation results in the horizontal and vertical directions, and rounding with a fixed offset value of 2; further including: S5.1, the instructions vr_out = _mx128_add_w(in_h_0,in_w_0) and vr_out = _mx128_add_w(vr_out,bias) are used to add the data in the in_h_0 register to the data in the in_w_0 register and to the data in the bias register, and then store the data in the vr_out register; where the bias register always contains the number 2. S5.2, then use the instructions vr_out = _mx128_srli_w(vr_out,2) and v16u8 out_value = _mx128_shufv(vr_out,vr_out,w_to_b) to shift the data in the vr_out register two bits to the right and convert the data from 32 bits to 8 bits, and store the data in the out_value register; S5.3 Finally, _mx128_su1q(out_value,dst_addr,0) is used to store the data in the out_value register into the output address dst_addr.

Images