Image data processing method, device, and medium based on homography transformation

Abstract

The method, device, electronic device, and medium for processing image data based on homography transformation include: a step (S101) of obtaining an initial homography transformation matrix corresponding to initial image data at a first viewpoint, where each initial element in the initial homography transformation matrix occupies a first number of bytes; a step (S103) of quantizing each initial element in the initial homography transformation matrix to generate a quantized homography transformation matrix, where each quantized element in the quantized homography transformation matrix occupies a second number of bytes, the second number of bytes being smaller than the first number of bytes; a step (S105) of generating a coordinate mapping matrix based on the quantized homography transformation matrix; and a step (S107) of obtaining target image data at a second viewpoint based on the initial image data and the coordinate mapping matrix. By generating the coordinate mapping matrix using the quantized homography matrix, which occupies a relatively small number of bytes after quantization, on the one hand, the space occupied by the quantized homography transformation matrix is ​​saved and on the other hand, the generation speed of the coordinate mapping matrix is ​​improved, thereby improving the overall efficiency of the image data processing process.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This disclosure claims priority to a Chinese patent application filed on November 21, 2022, bearing application number 202211459711.6 and entitled "Image data processing method, device and medium based on homography transformation," the entire contents of which are incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to the field of vision, and in particular to a method, apparatus, electronic device and computer-readable storage medium for processing image data based on homography transformation. [Background technology]

[0003] Homography transformation is usually used to convert input from one viewpoint into output from another viewpoint, and plays a very important role in image data processing fields such as image rectification, viewpoint transformation, image stitching and camera pose estimation, and simultaneous localization and mapping (SLAM).

[0004] Taking viewpoint transformation as an example, the homography transformation process first generates a coordinate mapping matrix based on a homography transformation matrix corresponding to initial image data at a certain viewpoint, and then removes index values ​​in the input data (i.e., initial image data) based on this coordinate mapping matrix to generate output data at a viewpoint different from the input data, thereby obtaining target image data at another viewpoint after transformation.In the related art, the homography transformation process involves using a homography transformation matrix with a data format of float32 to generate a coordinate mapping matrix with a data format of float32.

[0005] However, the process of generating the coordinate mapping matrix in the related art is relatively slow, which affects the efficiency of processing image data. Summary of the Invention [Problem to be solved by the invention]

[0006] In order to solve the above technical problem, the present disclosure provides an image data processing method, device, electronic device, and medium based on homography transformation. [Means for solving the problem]

[0007] An image data processing method based on homography transformation according to one aspect of the present disclosure includes: obtaining an initial homography transformation matrix corresponding to the initial image data at a first viewpoint, wherein each initial element in the initial homography transformation matrix occupies a first number of bytes; quantizing each initial element in the initial homography transform matrix to generate a quantized homography transform matrix, wherein each quantized element in the quantized homography transform matrix occupies a second number of bytes, the second number of bytes being smaller than the first number of bytes; generating a coordinate mapping matrix based on the quantized homography transformation matrix; and obtaining target image data at a second viewpoint based on the initial image data and the coordinate mapping matrix.

[0008] According to another aspect of the present disclosure, there is provided an image data processing apparatus based on homography transformation, comprising: an acquisition module for acquiring an initial homography transformation matrix corresponding to the initial image data at a first viewpoint, wherein each initial element in the initial homography transformation matrix occupies a first number of bytes; a quantization module for quantizing each initial element in the initial homography transform matrix to generate a quantized homography transform matrix, wherein each quantized element in the quantized homography transform matrix occupies a second number of bytes, the second number of bytes being smaller than the first number of bytes; A generation module for generating a coordinate mapping matrix based on the quantized homography transformation matrix, and determining target image data at a second viewpoint based on the initial image data and the coordinate mapping matrix.

[0009] A computer-readable storage medium according to a further aspect of an embodiment of the present disclosure stores a computer program for executing an image data processing method based on a homography transformation.

[0010] According to a further aspect of the present disclosure, there is provided an electronic device comprising: a processor; a memory for storing processor-executable instructions; The processor reads and executes executable instructions from the memory to implement the image data processing method based on the homography transformation.

[0011] A further aspect of an embodiment of the present disclosure is a computer program product, wherein instructions in the computer program product, when executed by a processor, cause the processor to perform the above-described homography transform-based image data processing method. [Effects of the Invention]

[0012] According to the above technical configuration, the embodiments of the present disclosure provide an image data processing method, device, electronic device, and computer-readable storage medium based on homography transformation, in which each initial element occupying a first number of bytes in an initial homography transformation matrix corresponding to initial image data at a first viewpoint is quantized into a quantized element occupying a second number of bytes to generate a quantized homography transformation matrix, i.e., the initial homography transformation matrix is ​​quantized into a quantized homography transformation matrix to reduce the occupied space of each quantized element in the quantized homography transformation matrix, and then a coordinate mapping matrix is ​​generated by the quantized homography transformation matrix. A sequence is generated, and then target image data at a second viewpoint is determined based on the initial image data and the coordinate mapping matrix. Compared to generating a coordinate mapping matrix using a homography transformation matrix formed with initial elements occupying a first number of bytes in the image data processing process of related technologies, the embodiment of the present disclosure generates a coordinate mapping matrix using a quantized homography matrix that occupies a relatively small number of bytes after quantization, thereby saving the space occupied by the quantized homography transformation matrix on the one hand and improving the speed of generating the coordinate mapping matrix on the other hand, thereby improving the overall efficiency of the image data processing process based on homography transformation. [Brief explanation of the drawings]

[0013] [Figure 1] 1A and 1B are schematic diagrams illustrating two types of viewpoints of image data in an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. [Figure 2] 1 is a schematic flowchart of an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. [Figure 3] FIG. 1 is a flow block diagram of generating a coordinate mapping matrix in an image data processing method based on a homography transformation according to an exemplary embodiment of the present disclosure. [Figure 4] 10 is another schematic flowchart of an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. [Figure 5]10 is a flowchart illustrating a process for generating a quantized homography transformation matrix in an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. [Figure 6] 10 is another flowchart of generating a coordinate mapping matrix in an image data processing method based on a homography transformation according to an exemplary embodiment of the present disclosure. [Figure 7] FIG. 7 is a flow block diagram corresponding to FIG. 6. [Figure 8] FIG. 10 is a further flow block diagram of generating a coordinate mapping matrix in an image data processing method based on a homography transform according to an exemplary embodiment of the present disclosure. [Figure 9] FIG. 1 is a structural diagram of an image data processing apparatus based on homography transformation according to an exemplary embodiment of the present disclosure; [Figure 10] FIG. 2 is another structural diagram of an image data processing apparatus based on homography transformation according to an exemplary embodiment of the present disclosure. [Figure 11] FIG. 2 is a structural diagram of a quantization module in an image data processing apparatus based on homography transformation according to an exemplary embodiment of the present disclosure. [Figure 12] 1 is a structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The described embodiments are only some of the embodiments of the present disclosure, and are not all of the embodiments of the present disclosure, and the present disclosure is not limited to the exemplary embodiments described herein.

[0015] Unless otherwise specified, the relative arrangement of components and steps, numerical expressions and values ​​described in these examples do not limit the scope of the present disclosure.

[0016] [Application Summary] In the related art, the data format of each data in the homography transformation matrix corresponding to the initial image data at a certain viewpoint is usually float32, that is, this homography transformation matrix is ​​stored in a computer program, and the storage format of each data therein is usually 32-bit float type binary storage, that is, the number of bits occupied by each data is 32 bits, and accordingly, the number of bytes occupied by each data is 4 bits. The homography transformation matrix H can be specifically expressed as the following matrix:

number

[0017] For example, in the homography transformation matrix H, h 11 is 5.0 in decimal, and its corresponding binary notation is 101.0, which is equivalent to 1.01 x 2^2, then the float32 data format is represented as 001111100010000000000000000000000, i.e., when the decimal number 5.0 is represented in the float32 data format, it occupies 32 bits, where the highest 1 bit is the sign bit (i.e., 0, corresponding to a positive number), the next 8 bits are the exponent (i.e., 01111100, corresponding to 2), and the remaining 23 are the significant digits (i.e., 010000000000000000000000, corresponding to 01).

[0018] When generating a coordinate mapping matrix using this homography transformation matrix H, arithmetic processing must be performed on each data in the homography transformation matrix H, and accordingly, arithmetic processing must be performed on each bit number in each data. This reduces the calculation speed of the homography transformation matrix H, i.e., the speed at which the coordinate mapping matrix is ​​generated using this homography transformation matrix H is relatively slow, and since target image data at other viewpoints is determined based on the coordinate mapping matrix, the efficiency of determining target image data at other viewpoints based on this homography transformation is reduced.

[0019] In view of this, the embodiments of the present disclosure provide an image data processing method, device, electronic device, and medium based on homography transformation, which improves the speed of generating a coordinate mapping matrix and improves the efficiency of processing image data.

[0020] [Example System] Homography transformation can be understood as simply describing the positional mapping relationship between the object's world coordinate system and the pixel coordinate system. Homography transformation plays a very important role in the fields of image correction, image-to-vision transformation, image stitching, camera pose estimation, visual SLAM, etc.

[0021] For example, in the homography transformation process, an image captured at a certain angle can be transformed to another viewpoint to achieve image viewpoint transformation. FIG. 1 is a schematic diagram of image data at two viewpoints in an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. Referring to FIG. 1, taking the viewpoint transformation as an example, when an image of the floor of an indoor corridor is photographed, the image within the black frame surrounded by sampling points P, Q, R, and S in FIG. 1(a) is an image captured at a first viewpoint, and the image within the black frame surrounded by sampling points P, Q, R, and S in FIG. 1(b) is an image at a second viewpoint (i.e., a plan view). In the image data processing process, the image data at the first viewpoint in FIG. 1(a) can be transformed into image data at the second viewpoint in FIG. 1(b) using the image data processing method based on homography transformation. That is, by transforming the image data at the first viewpoint (e.g., a side view) in FIG. 1(a) into image data at the second viewpoint (i.e., a plan view) in FIG. 1(b), image feature data of an object in a scene from different viewpoints can be easily observed.

[0022] Taking an example of an automated driving assistance system for automobiles, in order to adapt to the driver's observation habits, it is desirable to obtain a plan view of the entire vehicle exterior at a certain time and better combine it with image data obtained by a camera on the side of the vehicle to grasp the environment around the vehicle. However, in actual design, due to limitations in the shooting viewpoint and range of a camera used to observe the vehicle top, the captured image is usually taken from the side of the vehicle top (e.g., a first viewpoint) to obtain vehicle top image data from the first viewpoint. This vehicle top image data is not convenient for observing the entire vehicle top and violates the driver's observation habits. Taking these points into consideration, a homography transformation method can be used to convert the vehicle top image data from the first viewpoint into vehicle top image data from a second viewpoint (i.e., a vehicle top plan view), which can better capture the situation at each position of the vehicle top and better grasp the environment around the vehicle.

[0023] Taking an example of a warehouse robot photographing a warehouse shelf, the first viewpoint can be the viewpoint from which the warehouse robot photographs the shelf from a direction at a 45° angle from the front of the shelf, and the second viewpoint can be the viewpoint from which the warehouse robot photographs the shelf from the front of the shelf. In reality, there may be many shelves and miscellaneous items in a warehouse, and the warehouse robot cannot move to the front of the shelf to photograph information such as the tag content on the front of the shelf, but can only photograph the shelf from another angle via the warehouse robot. For example, the shelf can be photographed from the first viewpoint (e.g., a viewpoint from which the shelf is photographed from a direction at a 45° angle from the front of the shelf), and the first viewpoint can be converted into the second viewpoint (i.e., a viewpoint photographed from the front of the shelf) through a homography transformation process. In this way, related information such as the tag content on the front of the shelf can be intuitively obtained from the image at the second viewpoint, which provides convenience for collecting information on actual shelves and positioning the warehouse robot.

[0024] In practice, the image data processing process (e.g., visual transformation) based on homography transformation includes two steps: the first step is to generate a coordinate mapping matrix based on the homography transformation matrix corresponding to the initial image data at a first viewpoint, and the second step is to remove index values ​​in the input data based on the coordinate mapping matrix to finally generate output data, where the input data are the orientations of multiple sampling points in the initial image data at a first viewpoint (e.g., coordinate values ​​in a pixel coordinate system and corresponding image data), and the output data are the orientations of sampling points in the corresponding initial image data in the target image data at a second viewpoint (e.g., coordinate values ​​of each sampling point in a pixel coordinate system and corresponding image data).

[0025] JPEG2025537598000003.jpg22157

number

[0026] As can be seen, the homography transformation matrix H is a 3-by-3 matrix, and h 11 , h 12 , …, h 33 are the elements in the homography transformation matrix H, respectively.

[0027] In the image data processing method, device, electronic device, and computer-readable storage medium based on homography transformation according to the embodiments of the present disclosure, before generating a coordinate mapping matrix using a homography transformation matrix, an initial homography transformation matrix corresponding to the initial image data at a first viewpoint is first obtained, and each initial element occupying a first number of bytes in the initial homography transformation matrix is ​​quantized into a quantized element occupying a second number of bytes to generate a quantized homography transformation matrix, i.e., the initial homography transformation matrix is ​​quantized into a quantized homography transformation matrix to reduce the occupied space of each quantized element in the quantized homography transformation matrix. First, a coordinate mapping matrix is ​​generated using a quantized homography transformation matrix, and then the initial image data at the first viewpoint is transformed into the target image data at the second viewpoint using the coordinate mapping matrix. Compared with the related art in which the coordinate mapping matrix is ​​generated using a homography transformation matrix formed with initial elements occupying a first number of bytes, the embodiments of the present disclosure generate the coordinate mapping matrix using a quantized homography matrix that occupies a relatively small number of bytes after quantization, which on the one hand saves the space occupied by the quantized homography transformation matrix and on the other hand improves the speed of generating the coordinate mapping matrix, thereby improving the overall efficiency of the image data processing process.

[0028] [Exemplary Method] 2 is a flowchart of an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. This method can be applied to electronic devices, and as shown in FIGS. 1 and 2, the method includes the following steps:

[0029] In S101, an initial homography transformation matrix corresponding to the initial image data at the first viewpoint is obtained.

[0030] Here, the initial image data at the first viewpoint can refer to the image data within the black frame surrounded by sampling points P, Q, R, and S in (a) of Figure 1, and for example, the initial image data includes coordinate values ​​corresponding to the four sampling points P, Q, R, and S and their image data.

[0031] Here, each initial element in the initial homography transformation matrix occupies the first number of bytes.

[0032] In the embodiment of the present disclosure, the initial homography transformation matrix is ​​a homography transformation matrix calculated in advance based on coordinate values ​​of a plurality of different angles. If H1 represents the initial homography transformation matrix, it is as follows:

number

[0033] where h 11 , h 12 , h 13 , , h 33 are the initial elements in H1, the number of first bytes is 4, and the number of bits is 32.

[0034] Illustratively, each initial element in the initial homography transformation matrix H1 is represented in a float32 data format. For example, h 11 is 5.0 in decimal and its corresponding binary notation is 101.0, which is equivalent to 1.01 x 2^2, then the float32 data format is represented as 001111100010000000000000000000000, i.e., the decimal number 5.0 occupies 32 bits when represented in float32 data format, where the highest 1 bit is the sign bit (i.e., 0, corresponding to a positive number), the next 8 bits are the exponent (i.e., 01111100, corresponding to 2), and the remaining 23 are the significant digits (i.e., 01000000000000000000000, corresponding to 01).

[0035] As will be understood, data expressed in decimal numbers can be converted to corresponding binary data using conventional conversion methods, and the following description of each matrix will be given using data expressed in decimal numbers.

[0036] Specifically, the embodiment of the present disclosure will be described in detail using the following decimal matrix as an example of H1.

number

[0037] In the above example, when each initial element in H1 is represented in float32 data format, the number of first bytes occupied by each initial element is 4, and accordingly, when each initial element in H1 is represented in float32 data format, the number of first bits occupied by each initial element is 32.

[0038] In S103, each initial element in the initial homography transformation matrix is ​​quantized to generate a quantized homography transformation matrix.

[0039] Here, each quantized element in the quantized homography transform matrix occupies a second number of bytes, which is smaller than the first number of bytes.

[0040] If we use H2 to represent the quantized homography transformation matrix, it becomes as follows:

number

[0041] JPEG2025537598000008.jpg33157

[0042] In some examples, the data format of the quantization element can be int8, that is, each initial element is quantized in the int8 data format to generate a quantized homography transformation matrix H2, and the number of second bytes occupied by each quantized element in this quantized homography transformation matrix H2 is 1, that is, the number of bits occupied by each quantized element is 8.

[0043] For example, the initial homography transformation matrix H1 is quantized in the int8 data format to obtain a quantized homography transformation matrix H2 expressed in decimal notation as follows:

number

[0044] In some other examples, the data format of the quantization element can be int16, that is, each initial element is quantized in the data format of int16 to generate a quantized homography transformation matrix H2, and the number of second bytes occupied by each quantized element in this quantized homography transformation matrix H2 is 2, that is, the number of bits occupied by each quantized element is 16.

[0045] For example, the above initial homography transformation matrix H1 is quantized in the int16 data format to obtain a quantized homography transformation matrix H2 expressed in decimal notation as follows:

number

[0046] It should be noted that the embodiments of the present disclosure can further quantize each initial element in other data formats, without restricting the data format of the initial element quantization, and it is sufficient to ensure that in the generated quantized homography transformation matrix, the second number of bytes occupied by each quantized element is smaller than the first number of bytes, and accordingly, the second number of bytes is smaller than the first number of bytes.

[0047] In S105, a coordinate mapping matrix is ​​generated based on the quantized homography transformation matrix.

[0048] 3 is a flow block diagram of generating a coordinate mapping matrix in an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. Specifically, the embodiment of the present disclosure can generate a coordinate mapping matrix M1 based on the quantized homography transformation matrix H2 and a homography transformation constant (also referred to as a grid). To distinguish the homography transformation constant from the homography transformation constant in the following example, the homography transformation constant in this example can be a first homography transformation constant (hereinafter referred to as A), which can be a matrix of (H*W, 3). It should be understood that H is the number of rows in the output data, W is the number of columns in the output data, and 3 means that a certain row and column position both have three values. H and W can be determined when the output data requirements are provided.

[0049] For example, if the required number of rows of output data is 3 and the number of columns is 3, i.e., H=3 and W=3, the first homography transformation constant A is a (3*3,3) matrix, i.e., this first homography transformation constant A can be understood as a matrix with 9 rows and 3 columns. Naturally, this first homography transformation constant A(3*3,3) can also be understood as a matrix with 3 rows and 3 columns, and each position in this matrix has three numerical values, which can be specifically expressed as follows:

number

[0050] Here, the first value of each position in the first homography transformation constant A is the column number of the matrix in which this position is located, the second value is the row number of the matrix in which this position is located, and the third value is the complement value 1, which facilitates multiplication of the value of each position by the homography transformation matrix (3,3).

[0051] In the embodiment of the present disclosure, a specific process of generating the coordinate mapping matrix M1 based on the quantized homography transformation matrix H2 may be as follows: First, an intermediate homography transform matrix can be generated based on the quantized homography transform matrix H2 and the first homography transform constant A. For example, the quantized homography transform matrix H2 can be multiplied by the first homography transform constant A to obtain an intermediate homography transform matrix (H*W, 3).

[0052] For example, after multiplying the quantized homography transformation matrix H2 quantized in the above int8 data format by the first homography transformation constant A, the obtained intermediate homography transformation matrix E is as follows:

number

[0053] Next, a coordinate mapping matrix M can be generated based on the intermediate homography transformation matrix E. For example, two intermediate sub-matrices can be determined based on the intermediate homography transformation matrix E, where the first sub-matrix E1 of the two intermediate sub-matrices can be expressed as (H*W,2) and the second sub-matrix E2 can be expressed as (H*W,1), and then the first sub-matrix E1 and the second sub-matrix E2 are divided to finally obtain the coordinate mapping matrix M(H,W,2).

[0054] For example, the first sub-matrix E1 obtained after dividing the intermediate homography transformation matrix E is specifically as follows:

number

[0055] Specifically, the second sub-matrix E2 is as follows:

number

[0056] By dividing E1 and E2, the coordinate mapping matrix M1 finally obtained is as follows:

number

[0057] The specific process and calculation details for generating the coordinate mapping matrix M1 can be directly referred to in the related art, and therefore will not be described here.

[0058] In S107, the target image data at the second viewpoint is determined based on the initial image data and the coordinate mapping matrix.

[0059] For example, after the coordinate mapping matrix M1 is determined, index values ​​in the input data can be removed based on this coordinate mapping matrix M1 to finally obtain output data, that is, the input data is transformed into output data of another viewpoint by the image data processing method based on homography transformation of the embodiment of the present disclosure. As can be understood, the input data can include orientations (e.g., coordinate values ​​in a pixel coordinate system) of multiple sampling points in the initial image data at a first viewpoint and image data at these orientations. For example, referring to Figure 1, the initial image data has four sampling points P, Q, R, and S, and the input data includes coordinate values ​​in the pixel coordinate system of the sampling points P, Q, R, and S and image data corresponding to each of these points.

[0060] In addition, the output data includes the orientation of the sampling points in the corresponding initial image data in the target image data at the second viewpoint and the target image data. For example, the output data includes the coordinate values ​​of sampling points P, Q, R, and S at the second viewpoint and the image data of each sampling point, thereby determining the target image data at the second viewpoint, that is, the coordinate mapping matrix generated by the homography transformation matrix, which transforms the initial image data at the first viewpoint into the target image data at the second viewpoint.

[0061] The specific process of removing the index values ​​in the input data based on the coordinate mapping matrix M1 can be directly referred to in the related art, and the description thereof will be omitted here.

[0062] As can be seen from the above, the image data processing method based on homography transformation in the embodiment of the present disclosure reduces the occupied space of each quantized element in the quantized homography transformation matrix by quantizing each initial element occupying a first number of bytes in the initial homography transformation matrix into a quantized element occupying a second number of bytes, i.e., by quantizing the initial homography transformation matrix formed by each initial element (occupying the first number of bytes) into a quantized homography transformation matrix formed by quantized elements (smaller than the first number of bytes), and then generates a coordinate mapping matrix using the quantized homography transformation matrix. Finally, the target image data for the second viewpoint is generated using this coordinate mapping matrix and the initial image data at the first viewpoint.

[0063] Compared to the related art, which generates a coordinate mapping matrix using a homography transformation matrix formed with initial elements occupying a first number of bytes (e.g., 4, corresponding data format is float32), the embodiment of the present disclosure quantizes the initial homography transformation matrix occupying a first number of bytes to generate a quantized homography matrix, in which the quantized elements occupy a second number of bytes, which is smaller than the first number of bytes, and then uses the quantized homography matrix with a relatively small number of bytes after quantization to generate the coordinate mapping matrix. Subsequently, the target image data at the second viewpoint is determined based on the initial image data and the coordinate mapping matrix, thereby on the one hand saving the space occupied by the quantized homography transformation matrix and on the other hand improving the speed of generating the coordinate mapping matrix and improving the overall efficiency of the image data processing process based on this homography transformation.

[0064] 4 is another schematic flowchart of an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. Referring to FIG. 4, in a possible implementation, the image data processing method based on homography transformation according to the embodiment of the present disclosure may further include the following steps before the step of quantizing each initial element in the initial homography transformation matrix to generate a quantized homography transformation matrix:

[0065] In S102, the quantization coefficient is determined based on the second byte number.

[0066] As described above, the number of second bytes is determined by the data format of the quantization element, i.e., the manner in which the initial element is quantized. For example, if each initial element is quantized in the int8 data format and the number of second bytes occupied by each quantization element in the generated quantization homography transformation matrix H2 is 1 and the number of second bits is 8, the corresponding quantization coefficient can be determined based on the number of second bytes. Subsequently, each initial element is quantized using the quantization coefficient to obtain the corresponding quantization element, and the quantization homography transformation matrix H2 can be formed.

[0067] Illustratively, the specific steps of determining the quantization coefficient based on the second byte number are as follows:

[0068] In S1021, the quantization display range is determined based on the second byte number.

[0069] For example, when each initial element is quantized in the int8 data format, that is, when the second byte number is 1, the quantized display range is determined to be 0 to 255. When each initial element is quantized in the int16 data format, that is, when the second byte number is 2, the quantized display range is determined to be 0 to 65535.

[0070] In practice, after the data format is determined, the number of bytes occupied by the data represented by this data format is determined, and the maximum and minimum values ​​of the data represented by the data format can be obtained accordingly, thereby determining the quantization display range in this data format.

[0071] In S1022, an actual value range is determined based on the maximum initial element and the minimum initial element in the initial homography transformation matrix.

[0072] For example, in the initial homography transformation matrix H1, h11 is the maximum initial element, and h 12 If is the smallest initial element, the actual numerical range of this initial homography transformation matrix H1 is h 12 ~h 11 Taking the above specific example of H1 as an example, 8201.3 is the maximum initial element of H1, and 0 is the minimum initial element of H1. In this example, the actual value range of H1 is 0 to 8201.3.

[0073] In S1023, the quantization coefficient is determined based on the quantization display range and the actual value range.

[0074] In some examples, the quantization coefficient (hereinafter, a) may be the ratio of the width of the quantization display range to the width of the actual value range. Here, the width of the quantization display range is the difference between the upper and lower limits of the quantization display range. For example, if each initial element is quantized in the int8 data format, that is, the second byte number is 1 and the quantization display range is determined to be 0 to 255, the width of the quantization display range is 255-0=255.

[0075] For example, if each initial element is quantized in the int16 data format, that is, the number of the second byte is 2, and the quantized display range is determined to be 0 to 65535, the width of the quantized display range is 65535-0=65535.

[0076] In addition, the width of the actual value range may be the difference between the upper limit and the lower limit of the actual value range. 12 ~h 11 The actual range of values ​​is h 11 -h 12 For example, taking the specific example of the initial homography matrix H1 above as an example, the width of the actual numerical range of H1 is 8201.3.

[0077] Next, taking the above-mentioned specific example of the initial homography matrix H1 as an example, if each initial element in the initial homography matrix H1 needs to be quantized in the int8 data format, the quantization coefficient (hereinafter referred to as a1) can be (8201.3-0) / 255=32.1.

[0078] When each initial element in the initial homography matrix H1 needs to be quantized in the int16 data format, the quantization coefficient (hereinafter represented as a2) can be (8201.3-0) / 65535=0.125.

[0079] Of course, in other examples, the quantization coefficient may be obtained by other calculation methods between the width of the quantization display range and the width of the actual value range, and the method for determining the quantization coefficient is not limited here.

[0080] In the embodiment of the present disclosure, after the quantization coefficient a is determined, each initial element in the initial homography transformation matrix H1 can be quantized based on this quantization coefficient a to generate a quantized homography transformation matrix H2.

[0081] JPEG2025537598000016.jpg34157

[0082] JPEG2025537598000017.jpg46157

[0083] For example, the initial homography transformation matrix H1 is quantized in the int8 data format to obtain a quantized homography transformation matrix H2 expressed in decimal notation as follows:

number

[0084] where H 1min Each data is the minimum value in H1. For example,

number

[0085] Furthermore, for example, the initial homography transformation matrix H1 is quantized in the int16 data format to obtain a quantized homography transformation matrix H2 expressed in decimal numbers as follows:

number

[0086] In other examples, other operations can be performed on the quantization coefficient a and each initial element in the initial homography transformation matrix H1 to generate corresponding quantization elements in the quantization homography transformation matrix H2, and the calculation method of the quantization elements is not limited here.

[0087] As can be understood, in the above example, after the initial homography transformation matrix H1 is obtained, each initial element in this initial homography transformation matrix H1 can be directly quantized, that is, in S102, there is no need to perform other processing on the initial homography matrix H1, and the quantization of each initial element in the initial homography matrix H1 is directly started to generate the quantized homography transformation matrix H2.

[0088] JPEG2025537598000021.jpg51157

[0089] Next, dequantization is performed using the quantized homography transformation matrix H1 quantized in the int8 data format as an example, and the dequantized matrix H3 is obtained as follows.

number

[0090] By comparing H3 with the initial homography transformation matrix H1, h 31 is 0.4, h 32 is 0.2, which is the reason why h 31 and h 32were found to be 0, resulting in an accuracy loss of (0.4-0) / 0.4=100%. This will affect the accuracy of the coordinate mapping matrix obtained based on the quantized homography transformation matrix H2, and thus the accuracy of the final target image data at the second viewpoint.

[0091] 5 is a flowchart illustrating a process for generating a quantized homography transformation matrix in an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. Referring to FIG. 5, in order to solve the above problem, in some other examples, step S103 of quantizing each initial element in the initial homography transformation matrix to generate a quantized homography transformation matrix specifically includes the following steps:

[0092] In S1032, the initial homography transformation matrix is ​​divided to generate at least two sub-initial homography transformation arrays.

[0093] Specifically, the initial homography transformation matrix H1 is divided into multiple sub-initial homography transformation arrays H 1a where the sub-initial homography transformation array H 1a The number of sub-initial homography transformation arrays H can be any integer between 2 and 9. For example, the initial homography transformation matrix H1 can be divided into 2, 4, 6, 8, or 9 sub-initial homography transformation arrays H 1a can be generated.

[0094] For example, based on the initial homography transformation matrix H1, multiple sub-initial homography transformation arrays H 1a Here, each sub-initial homography transformation array H 1a The initial elements in any one row in the initial homography transformation matrix H1 include at least some of the initial elements in at least one row in the initial homography transformation matrix H2, and each sub-initial homography transformation array H 1a The column order of at least some of the initial elements included in the initial homography transformation matrix H1 is the same as the column order of the initial homography transformation matrix H2, and each sub-initial homography transformation array H1a If includes initial elements of at least two rows, the initial elements of two adjacent rows are also adjacent in the initial homography transformation matrix H1.

[0095] Similarly, each sub-initial homography transformation array H 1a The initial elements in any one column in the initial homography transformation matrix H1 include at least some of the initial elements in at least one column in the initial homography transformation matrix H2, and each sub-initial homography transformation array H 1a The row order of at least some of the initial elements included in the initial homography transformation matrix H1 is the same as the row order of the initial homography transformation matrix H2, and each sub-initial homography transformation array H 1a If includes initial elements of at least two columns, the initial elements of two adjacent columns are also adjacent in the initial homography transformation matrix H1.

[0096] Each initial element in the initial homography transformation matrix H1 is a sub-initial homography transformation array H 1a is located.

[0097] For example, the initial homography transformation matrix H1 is divided along the column direction of the initial homography transformation matrix H1 to generate at least two sub-initial homography transformation arrays H 1a can generate at least two sub-initial homography transformation arrays H 1a is the first sub-initial homography transformation array H 11 and the second sub-initial homography transformation array H 12 Includes:

[0098] For example, at least two sub-initial homography transformation arrays H are generated based on each initial element row of the initial homography transformation matrix H1. 1a Here, each sub-initial homography transformation array H 1a contains at least one row of the initial element in the initial homography transformation matrix H1 and the same sub-initial homography transformation array H 1aThe two adjacent initial element rows included in are adjacent in the initial homography transformation matrix H1 and are included in different sub-initial homography transformation arrays H 1a The initial element rows contained in the initial homography transformation matrix H1 do not overlap, and each initial element row in the initial homography transformation matrix H1 is included in one sub-initial homography transformation array H 1a In other words, based on some initial element rows of the initial homography transformation matrix H1, the first sub-initial homography transformation array H 11 and determine each initial element row of the initial homography transformation matrix H1, and then generate a second sub-initial homography transformation array H1 according to some other initial element rows of the initial homography transformation matrix H1. 12 Determine each initial element row of .

[0099] Hereinafter, the initial homography transformation matrix H1 is divided along the column direction of the initial homography transformation matrix H1 to generate two sub-initial homography transformation arrays H 1a and generate the first sub-initial homography transformation array H 11 and the second sub-initial homography transformation array H 12 For example,

[0100] Illustratively, a first sub-initial homography transformation array H is generated based on the first and second initial element rows of the initial homography transformation matrix H1. 11 , and based on the initial element row of the third row of the initial homography transformation matrix H1, a second sub-initial homography transformation array H 12 Determine each initial element row of .

[0101] For example, the first sub-initial homography transformation array H 11 can be a 2-by-3 array, i.e., H 11 (2,3) can be specifically expressed as follows:

number

[0102] where H 11can be an array formed by the first and second rows of the initial homography transformation matrix H1, i.e., m 11 =h 11 , m 12 =h 12 , m 13 =h 13 , m 21 =h 21 , m 22 =h 22 , m 23 =h 23 , is.

[0103] Second sub-initial homography transformation array H 12 can be a 1-by-3 array, i.e., H 12 (1,3) can be specifically expressed as follows:

number

[0104] where H 12 may be an array formed by the third row of the initial homography transformation matrix H1, i.e., m 11 =h 31 , m 12 =h 32 , m 13 =h 33 is.

[0105] Next, using the initial homography transformation matrix H1 as an example, the first sub-initial homography transformation array H obtained by performing the above division on the initial homography transformation array H1 is obtained. 11 (2,3) is specifically as follows:

number

[0106] Second sub-initial homography transformation array H 12 (3,1) is specifically as follows:

number

[0107] In S1034, each initial element in each sub-initial homography transform array is quantized to generate a corresponding sub-quantized homography transform array.

[0108] Specifically, after the execution of S1032 is completed, each sub-initial homography transformation array H 1a quantize each initial element in 1a can be generated.

[0109] As will be appreciated, similar to the process of quantizing each initial element in the initial homography transformation matrix H described above, in some examples, each sub-initial homography transformation array H 1a quantizing each initial element in {circumflex over (x)} to generate each corresponding sub-quantized homography transform array, the following steps may be included.

[0110] In S1031a, the quantization display range is determined based on the second byte number.

[0111] For example, when each initial element is quantized in the int8 data format, that is, when the second byte number is 1, the quantized display range is determined to be 0 to 255. When each initial element is quantized in the int16 data format, that is, when the second byte number is 2, the quantized display range is determined to be 0 to 65535.

[0112] As can be seen from the above, after the quantized data format is determined, the number of bytes occupied by the data (i.e., quantization elements) represented by this data format is determined, and the maximum and minimum values ​​of the data represented by the data format can be obtained accordingly, thereby determining the quantization display range in this data format.

[0113] In some examples, each sub-initial homography transformation array H 1aThe quantization element formats of the sub-initial homography transformation arrays H 1a are quantized in the same data format to generate the corresponding sub-quantized homography transformation array H 2a and generate the sub-quantized homography transformation array H 2a Each quantization element in has the same data format. Thus, in this example, each sub-initial homography transformation array H 1a The quantized display ranges of the are the same.

[0114] For example, each sub-initial homography transformation array H 1a quantization is realized in the int8 data format, that is, each sub-initial homography transformation array H 1a quantize each initial element of 2a and generate the initial homography transformation array H 1a The quantized display range for quantizing is 0 to 255.

[0115] In some other examples, each sub-initial homography transformation array H 1a The quantization element formats of the sub-initial homography transformation array H may not be identical. For example, 1a quantized into one data format (e.g., int8) to generate the corresponding sub-quantized homography transform array H 2a and generate at least another sub-initial homography transformation array H 1a is quantized into another data format (e.g., int16) to obtain the corresponding sub-quantized homography transformation array H 2a In this example, we generate the sub-initial homography transformation array H 1a The quantization display range is different when quantizing

[0116] Illustratively, at least one sub-initial homography transformation array H 1aquantization is realized in the int8 data format, that is, this one sub-initial homography transformation array H 1a Each initial element of is quantized, and the quantized display range is 0 to 255. For example, the first sub-initial homography transformation array H 11 quantization is realized in the int8 data format, that is, the first sub-initial homography transformation array H 11 Each initial element is quantized, and the quantized display range is 0 to 255.

[0117] At least one more sub-initial homography transformation array H 1a quantization is realized in the int16 data format, that is, this sub-initial homography transformation array H 1a Each initial element of is quantized, and the quantized display range is 0 to 65535. For example, the second sub-initial homography transformation array H 12 quantization is realized in the int16 data format, and the quantized display range is 0 to 65535.

[0118] In S1031b, each sub-initial homography transformation array H 1a The sub-actual numerical range of each sub-initial homography transformation array is determined based on the maximum initial element and the minimum initial element in .

[0119] For example, the first sub-initial homography transformation array H 11 In m 11 is the maximum initial element, and m 12 is the smallest initial element, this first sub-initial homography transformation array H 11 The sub-actual numerical range of m 12 ~m 11 For example, the first sub-initial homography transformation array H 11 In this case, 8201.3 is H 11 is the largest initial element, and 0 is H 11 In this example, H 11The sub-actual numeric range is 0 to 8201.3.

[0120] Second sub-initial homography transformation array H 12 In m 11 is the maximum initial element, and m 13 is the smallest initial element, this second sub-initial homography transformation array H 12 The sub-actual numerical range of m 11 ~m 12 For example, the second sub-initial homography transformation array H 12 In this case, 69.8 is H 12 The maximum initial element is 0.2, and H 12 In this example, H 12 The sub-actual numerical range is 0.2 to 69.8.

[0121] In S1031c, each sub-initial homography transformation array H is calculated based on the quantized display range and the sub-actual numerical range. 1a The sub-quantized coefficients of are determined.

[0122] In some examples, the sub-quantization factor (which may be denoted hereinafter as b) may be the ratio of the width of the quantization display range to the width of the sub-actual value range.

[0123] Here, the width of the quantization display range is the difference between the upper and lower limits of the quantization display range. For example, the sub-initial homography transformation array H 1a quantize the initial element in , that is, if the number of the second byte is 1 and the quantization display range is determined to be 0 to 255, the width of the quantization display range is 255-0=255. Also, for example, the corresponding sub-initial homography transformation array H 1a is quantized, that is, the number of the second byte is 2, and the quantized display range is determined to be 0 to 65535, the width of the quantized display range is 65535.

[0124] In addition, the width of the sub-actual numerical range may be the difference between the upper limit value and the lower limit value of the sub-actual numerical range. 11 The actual range of values ​​is m 12 ~m 11 If this first sub-initial homography transformation array H 11 The width of the sub-actual numerical range is m 11 -m 12 For example, the first sub-initial homography transformation array H 11 The width of the sub-actual numeric range is 8201.3.

[0125] In this way, the first sub-initial homography transformation array H 11 If each initial element in needs to be quantized, the sub-quantization coefficient b (e.g., b1) can be (8201.3-0) / 255=32.1. 12 If each initial element in needs to be quantized, the sub-quantization coefficient b (eg, b2) can be (69.8-0.2) / 255=0.27.

[0126] Similarly, the first sub-initial homography transformation array H 11 If each initial element in needs to be quantized, the sub-quantization coefficient b1′ can be (8201.3−0) / 65535=0.13. 12 If each initial element in needs to be quantized, the sub-quantization coefficient b2' can be (69.8-0.2) / 65535=0.001.

[0127] Of course, in other examples, the sub-quantization coefficients may be obtained by other calculation methods (e.g., weighted average, variance, etc.) between the width of the quantization display range and the width of the sub-actual value range, and the calculation method of the quantization coefficients is not limited here.

[0128] Each sub-initial homography transformation array H1a After the sub-quantization coefficient b is determined, in the embodiment of the present disclosure, each sub-initial homography transformation array H 1a quantize each initial element in 2a can be generated.

[0129] For example, the first sub-initial homography transformation array H 11 is quantized to obtain the first sub-quantized homography transformation array H 21 Specifically, the first sub-initial homography transformation array H 11 quantize each initial element in 21 , i.e., the first sub-quantized homography transformation array H 21 Each quantized element in is obtained by quantizing the corresponding initial element, where H 21 Specifically, it can be expressed as follows:

number

[0130] Furthermore, for example, the second sub-initial homography transformation array H 12 is quantized to obtain the second sub-quantized homography transformation array H 22 Specifically, the second sub-initial homography transformation array H 12 quantize each initial element in 22 , i.e., the second sub-quantized homography transformation array H 22 Each quantized element in is obtained by quantizing the corresponding initial element, where H 22 Specifically, it can be expressed as follows:

number

[0131] Specifically, each sub-initial homography transformation array H 1a When quantizing each initial element in 1a In each initial element h 1a is (h 1a -h 1amin ) / b, we use the sub-quantized homography transformation matrix H 2a The corresponding quantized element h in 2a We obtain h 2a =(h 1a -h 1amin ) / b, and accordingly, H 2a =(H 1a -H 1amin ) / b, where h 1amin is the sub-initial homography transformation array H 1a is the minimum value of each initial element in H 1amin is, h 1amin is an array formed by H 1amin All data in h 1amin is.

[0132] For example, in the int8 data format, the first sub-initial homography transformation array H 11 is quantized to obtain the first sub-quantized homography transformation array H 21 is obtained as follows:

number

[0133] where H 11min is, h 11min is an array formed by H 11min All data in h 11min (First sub-initial homography transformation array H 11 (minimum value of each initial element in

[0134] The second sub-initial homography transformation matrix H above is in int8 data format. 12is quantized to obtain the second sub-quantized homography transformation array H 22 is obtained as follows:

number

[0135] where H 12amin is, h 12min is an array formed by H 12min All data in h 12min (Second sub-initial homography transformation array H 12 (minimum value of each initial element in

[0136] Furthermore, for example, in the int16 data format, the first sub-initial homography transformation matrix H 11 is quantized to obtain the first sub-quantized homography transformation array H 21 is obtained as follows:

number

[0137] The above second sub-initial homography transformation matrix H in int16 data format 12 is quantized to obtain the second sub-quantized homography transformation array H 22 is obtained as follows:

number

[0138] In S1036, a quantized homography transform matrix is ​​generated based on the plurality of sub-quantized homography transform matrices.

[0139] Specifically, the quantized homography transformation matrix H2 is a matrix of each sub-quantized homography transformation array H 2a For example, the quantized homography transformation matrix H2 includes the first sub-quantized homography transformation array H 21 and the second sub-quantized homography transformation array H22 Includes:

[0140] As can be seen, based on the initial homography transformation matrix H1, each sub-initial homography transformation array H 1a and each sub-initial homography transformation array H 1a By quantizing each of these, the accuracy of the finally generated quantized homography transformation matrix H2 is improved, and this accuracy is not lost in the process of quantizing the initial homography transformation matrix H1.

[0141] For example, the first sub-quantized homography transformation array H obtained by quantizing in the int8 data format 21 and the second sub-quantized homography transformation array H 22 are dequantized, and the two dequantized subarrays obtained by the dequantization are column-stitched. The matrix after stitching is compared with the initial homography transformation matrix H1 to analyze the accuracy loss.

[0142] Illustratively, the first sub-quantized homography transform array H 21 quantized element h in 21a is, h 21a *b1+h 11min Through the calculation of the inverse quantization formula, each inverse quantization element h4 in the inverse quantization array H4 can be obtained, that is, h4=h 21a *b1+h 11min The finally obtained dequantized array H4 is as follows:

number

[0143] Similarly, the second subquantized homography transform array H 22 quantized element h in 22a is, h 22a *b2+h 12 Through the calculation of the inverse quantization formula min, each inverse quantization element h5 in the inverse quantization array H5 can be obtained, that is, h5=h 22a *b2+h 12min, and the finally obtained dequantized array H5 is as follows:

number

[0144] Then, the inverse quantized array H4 and the inverse quantized array H5 are stitched together through the reverse process of the division method of the initial homography transformation matrix H1 to obtain a matrix H6 as follows:

number

[0145] Comparing matrix H6 with the initial homography transformation matrix H1 shows that no accuracy is lost.

[0146] Multiple subquantized homography transform array H 2a After generating the quantized homography transformation matrix H2 by: ##EQU1## that is, after step S2036 is completed, the following steps may be included:

[0147] In S205, a plurality of sub-quantized homography transformation arrays H in the quantized homography transformation matrix H2 are 2a Based on this, a coordinate mapping matrix M is generated.

[0148] As can be seen from the above example, the initial homography transformation matrix H1 is divided into multiple sub-initial homography transformation arrays H 1a For example, the initial homography transformation matrix H1 is divided into a first sub-initial homography transformation array H1 and a second sub-initial homography transformation array H 12 and divide it into the first sub-initial homography transformation array H 11 and the second sub-initial homography transformation array H 12 The first sub-quantized homography transformation array H 21 and the second sub-quantized homography transformation array H 22There is no loss of precision in the generated coordinate mapping matrix, thereby improving the accuracy of the output data (i.e., target image data).

[0149] Fig. 6 is another flowchart of generating a coordinate mapping matrix in an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure, and Fig. 7 is a flow block diagram corresponding to Fig. 6. Referring to Fig. 6 and Fig. 7, in a possible embodiment, S205 specifically may include the following steps:

[0150] In S2051, the homography transform constant and the first sub-quantized homography transform array H 21 Generate a first intermediate quantized homography transform array based on:

[0151] To distinguish from the first homography transformation constant A in the above example, in this example the homography transformation constant can be a second homography transformation constant (which can be represented as B).

[0152] In some alternative examples, the first sub-quantized homography transformation array H 21 and a second homography transform constant B to generate a first intermediate quantized homography transform array H. As will be appreciated, in this example, the first sub-quantized homography transform array H 21 and the second homography transform constant B, the first sub-quantized homography transform array H 21 The number of columns of the second homography transformation constant B must be equal to the number of rows of the second homography transformation constant B.

[0153] Illustratively, the first sub-quantized homography transform array H 21 has three columns and the second homography transformation constant B has three rows, then in this second homography transformation constant B(H*W,3), H=3 and W=3, and this second homography transformation constant B can be expressed as follows:

number

[0154] For example, the first sub-quantized homography transform array H 21 Specifically, the first intermediate quantized homography transform array F1(H*W,2) obtained by multiplying H*W by the second homography transform constant B is as follows:

number

[0155] In S2053, the homography transform constant and the second sub-quantized homography transform array H 22 Generate a second intermediate quantized homography transform array based on:

[0156] In some alternative examples, the second sub-quantized homography transform array H 22 and the homography transform constant to generate a second intermediate quantized homography transform array.

[0157] As can be seen, in this example, the second sub-quantized homography transform array H 22 and the homography transform constant, the second sub-quantized homography transform array H 22 The number of columns of must be equal to the number of rows of this homography transformation constant.

[0158] Illustratively, the second sub-quantized homography transform array H 22 is 3 columns and the number of rows of the homography transformation is 3, this homography transformation constant can be the second homography transformation constant B(3*3,3).

[0159] Next, the second subquantized homography transform array H in the int8 data format 22 For example, the second sub-quantized homography transformation array H 22Specifically, the second intermediate quantized homography transform array F2 obtained by multiplying by the second homography transform constant B is as follows:

number

[0160] In S2055, a coordinate mapping matrix M2 is generated based on the first intermediate quantized homography transform array and the second intermediate quantized homography transform array.

[0161] In some possible implementations, the coordinate mapping matrix M2 can be obtained by dividing the first intermediate quantized homography transform array and the second intermediate quantized homography transform array.

[0162] For example, the above F1 and F2 can be divided, and the final coordinate mapping matrix M2 is as follows:

number

[0163] In addition, the coordinate mapping matrix M0 can be generated based on the initial homography transformation matrix H1 and the homography transformation constant (also called grid) by adopting the flow for generating the coordinate mapping matrix shown in FIG. 2, as follows:

number

[0164] By comparing M2 and M1 with M0, it is found that M2 is closer to M0, and the first sub-initial homography transformation array H is calculated based on the initial homography transformation matrix H1. 11 and the second sub-initial homography transformation array H 12 and the first sub-initial homography transformation array H 11 and the second sub-initial homography transformation array H 12is quantized to obtain a first sub-quantized homography transformation array H 21 and the second sub-quantized homography transformation array H 22 and accordingly generates the first sub-quantized homography transformation array H 21 and the second sub-quantized homography transformation array H 22 It can be seen that the accuracy of the coordinate mapping matrix generated by

[0165] In some other examples, the initial homography transformation matrix H1 is divided into four sub-initial homography transformation arrays H 1a In other words, based on the initial homography transformation matrix H1, four sub-initial homography transformation arrays H 1a can be determined.

[0166] In some examples, the first sub-initial homography transformation array H 11 is quantized to obtain the first sub-quantized homography transformation array H 12 Before generating the first sub-initial homography transformation array H 11 The first sub-initial homography transformation array H 11 is divided into multiple third sub-initial homography transformation arrays H 13 The method may include generating:

[0167] For example, the first sub-initial homography transformation array H 11 Based on each initial element sequence, a plurality of third sub-initial homography transformation arrays H 13 Here, each third sub-initial homography transformation array H 13 is the first sub-initial homography transformation array H 11 and the same third sub-initial homography transformation array H 13 The two adjacent initial element sequences included in the first sub-initial homography transformation array H 11 The third sub-initial homography transformation array H 13The initial element sequences included in the first sub-initial homography transformation array H 11 Each initial element sequence in 13 is located.

[0168] Here, a plurality of third sub-initial homography transformation arrays H 13 is the fourth sub-initial homography transformation array H 14 and the fifth sub-initial homography transformation array H 15 For example, the first sub-initial homography transformation array H 11 Based on the initial element sequence of some of the elements, the fourth sub-initial homography transformation array H 14 Each initial element sequence is determined, and the first sub-initial homography transformation array H 11 Based on the other initial element sequences, the fifth sub-initial homography transformation array H 15 Each initial element sequence can be determined.

[0169] For example, the first sub-initial homography transformation array H 11 along the row direction of the first sub-initial homography transformation array H 11 are divided into two third sub-initial homography transformation arrays H 13 can be divided into four sub-initial homography transformation arrays H 14 and the fifth sub-initial homography transformation array H 15 is.

[0170] In other words, the first sub-initial homography transformation array H 11 Based on the first and second columns of the initial element sequence, a fourth sub-initial homography transformation array H 14 Each initial element sequence is determined, and the first sub-initial homography transformation array H 11 Based on the initial element sequence of the third column of the fifth sub-initial homography transformation array H 15 The initial element sequence can be determined.

[0171] For example, the fourth sub-initial homography transformation array H 14can be a 2-by-2 array, i.e., H 14 Specifically, (2,2) can be expressed as follows:

number

[0172] Fifth sub-initial homography transformation array H 15 can be a 2-by-1 array, i.e., H 15 Specifically, (2,1) can be expressed as follows:

number

[0173] Next, the first sub-initial homography transformation matrix H 11 For example, the first sub-initial homography transformation matrix H 11 The fourth sub-initial homography transformation array H obtained by dividing 14 (2,2) is specifically as follows:

number

[0174] First sub-initial homography transformation matrix H 11 The fifth sub-initial homography transformation array H obtained by dividing 15 (2,1) is specifically as follows:

number

[0175] Furthermore, for example, the second sub-initial homography transformation array H 12 is quantized to obtain the second sub-quantized homography transformation array H 22 Before generating the second sub-initial homography transformation array H 12 The second sub-initial homography transformation array H along the row direction of 12is divided into multiple sixth sub-initial homography transformation arrays H 16 where the sixth sub-initial homography transformation array H 16 is the 7th sub-initial homography transformation array H 17 and the 8th sub-initial homography transformation array H 18 may include:

[0176] For example, the second sub-initial homography transformation array H 12 Based on each initial element sequence, a plurality of sixth sub-initial homography transformation arrays H 16 Here, each sixth sub-initial homography transformation array H 16 is at least one column of the second sub-initial homography transformation array H 12 and the same sixth sub-initial homography transformation array H 16 The two adjacent initial element sequences contained in are the second sub-initial homography transformation array H 12 The sixth sub-initial homography transformation array H 16 The initial element sequences contained in the second sub-initial homography transformation array H 12 Each initial element sequence in 16 is located.

[0177] Illustratively, the second sub-initial homography transformation array H 12 Based on the initial element sequence of some of the elements, the seventh sub-initial homography transformation array H 17 Each initial element sequence is determined, and the second sub-initial homography transformation array H 12 Based on the other initial element sequences, the 8th sub-initial homography transformation array H 18 can be determined.

[0178] For example, the second sub-initial homography transformation array H 12 Along the row direction of the second sub-initial homography transformation array H 12 Two sixth sub-initial homography transformation arrays H 16can be divided into seventh sub-initial homography transformation arrays H 17 and the 8th sub-initial homography transformation array H 18 is.

[0179] In other words, the second sub-initial homography transformation array H 12 Based on the first and second columns of the initial element sequence, the seventh sub-initial homography transformation array H 17 Each initial element sequence is determined, and the second sub-initial homography transformation array H 12 Based on the initial element sequence of the third column of the eighth sub-initial homography transformation array H 18 The initial element sequence can be determined.

[0180] Illustratively, the seventh sub-initial homography transformation array H 17 can be a 1-by-2 array, i.e., H 17 (1,2) can be specifically expressed as follows:

number

[0181] 8th sub-initial homography transformation array H 18 may be a 1-by-1 array, i.e., H 18 Specifically, (1,1) can be expressed as follows:

number

[0182] Next, the second sub-initial homography transformation matrix H 12 For example, the second sub-initial homography transformation matrix H 12 The seventh sub-initial homography transformation array H obtained by dividing 17 (1,2) is specifically as follows:

number

[0183] Second sub-initial homography transformation matrix H 12 The 8th sub-initial homography transformation array H obtained by dividing 18 (1,1) is specifically as follows:

number

[0184] Accordingly, the step of quantizing the first sub-initial homography transform array to generate a first sub-quantized homography transform array may include the steps of quantizing the fourth sub-initial homography transform array to generate a fourth quantized homography transform array, and quantizing the fifth sub-initial homography transform array to generate a fifth quantized homography transform array.

[0185] Here, the fourth sub-initial homography transformation array H 14 When quantizing, first, the fourth sub-initial homography transformation array H 14 A sub-quantized coefficient b (for example, b4) of the quantized coefficient b can be determined.

[0186] Specifically, the fourth sub-initial homography transformation array H 14 In m 11 is the maximum initial element, and m 12 is the smallest initial element, this fourth sub-initial homography transformation array H 14 The sub-actual numerical range of m 12 ~m 11 For example, the fourth sub-initial homography transformation array H 14 In this case, 485.9 is H 14 is the largest initial element, and 0 is H 14 In this example, H 14 The sub-actual numeric range is 0 to 485.9.

[0187] The fourth sub-initial homography transformation array H in int8 data format 14If each initial element in needs to be quantized, the sub-quantization coefficient b4 can be (485.9-0) / 255=1.9. 14 If each initial element in needs to be quantized, the sub-quantization coefficient b4' can be (485.9-0) / 65535=0.007.

[0188] Fourth sub-initial homography transformation array H 14 After the quantization coefficients b (for example, b4 and b4') are determined, the fourth sub-initial homography transformation array H 14 can be quantized.

[0189] For example, in the int8 data format, the above fourth sub-initial homography transformation array H 14 is quantized to obtain the fourth sub-quantized homography transformation array H 24 is obtained as follows:

number

[0190] where H 14min is, h 14min is an array formed by H 14min All data in h 14min (Fourth sub-initial homography transformation array H 14 (minimum value of each initial element in

[0191] The above fourth sub-initial homography transformation array H in int16 data format 14 is quantized to obtain the fourth sub-quantized homography transformation array H 24 (2,2) is obtained as follows:

number

[0192] Similarly, the fifth sub-initial homography transformation array H 15 When quantizing, first, the fifth sub-initial homography transformation array H 15 A sub-quantized coefficient b (eg, b5) of the vector quantization coefficient b can be determined.

[0193] Specifically, the fifth sub-initial homography transformation array H 15 In m 13 is the maximum initial element, and m 23 is the smallest initial element, this fifth sub-initial homography transformation array H 15 The sub-actual numerical range of m 23 ~m 13 For example, the above fifth sub-initial homography transformation array H 15 In this case, 8201.3 is H 15 The largest initial element is 8120.2, and H 15 In this example, H 15 The sub-actual numerical range is 8120.2 to 8201.3.

[0194] In this way, the fifth sub-initial homography transformation array H 15 If each initial element in needs to be quantized, the sub-quantization coefficient b5 can be (8201.3-8120.2) / 255=0.31. 15 If each initial element in needs to be quantized, the sub-quantization coefficient b5' can be (8201.3-8120.2) / 65535=0.001.

[0195] Fifth sub-initial homography transformation array H 15 After the sub-quantization coefficients b (for example, b5 and b5') are determined, the fifth sub-initial homography transformation array H 15 can be quantized.

[0196] For example, in the int8 data format, the above fifth sub-initial homography transformation array H15 is quantized to obtain the fifth sub-quantized homography transformation array H 25 is obtained as follows:

number

[0197] where H 15min h 15min is an array formed by H 15min All data in h 15min (5th sub-initial homography transformation array H 15 (minimum value of each initial element in

[0198] The above fifth sub-initial homography transformation array H in int16 data format 15 is quantized to obtain the fifth sub-quantized homography transformation array H 25 is obtained as follows:

number

[0199] Also, in some examples, the step of quantizing the second sub-initial homography transform array to generate the second sub-quantized homography transform array may include the steps of quantizing the seventh sub-initial homography transform array to generate a seventh quantized homography transform array, and quantizing the eighth sub-initial homography transform array to generate an eighth quantized homography transform array.

[0200] Here, the seventh sub-initial homography transformation array H 17 When quantizing, first, the seventh sub-initial homography transformation array H 17 A sub-quantized coefficient b (eg, b7) of the quantized coefficient b can be determined.

[0201] Specifically, the seventh sub-initial homography transformation array H 17 In m 12 is the maximum initial element, and m11 is the smallest initial element, the seventh sub-initial homography transformation array H 17 The sub-actual numerical range of m 12 ~m 11 For example, the seventh sub-initial homography transformation array H 17 In this case, 0.4 is H 17 The maximum initial element is 0.2, and H 17 In this example, H 17 The sub-actual numerical range is 0.2 to 0.4.

[0202] The seventh sub-initial homography transformation array H in int8 data format 17 If each initial element in needs to be quantized, the sub-quantization coefficient b7 can be (0.4-0.2) / 255=0.00078. 17 If each initial element in needs to be quantized, the sub-quantization coefficient b7' can be (0.4-0.2) / 65535=0.000003.

[0203] 7th sub-initial homography transformation array H 17 After the sub-quantization coefficients b (for example, b7 and b7') are determined, the seventh sub-initial homography transformation array H 17 can be quantized.

[0204] For example, in the int8 data format, the above seventh sub-initial homography transformation array H 17 is quantized to obtain the seventh sub-quantized homography transformation array H 27 is obtained as follows:

number

[0205] where H 17min h 17min is an array formed by H 17minAll data in h 17min (7th sub-initial homography transformation array H 17 (minimum value of each initial element in

[0206] The above seventh sub-initial homography transformation array H in int16 data format 17 is quantized to obtain the seventh sub-quantized homography transformation array H 27 is obtained as follows:

number

[0207] Similarly, the 8th sub-initial homography transformation array H 18 When quantizing, first, the 8th sub-initial homography transformation array H 18 A sub-quantized coefficient b (for example, b8) can be determined.

[0208] Specifically, the 8th sub-initial homography transformation array H 18 In this case, data m 13 Since there is only one data m 13 H 18 The maximum initial element and the minimum initial element of this 8th sub-initial homography transformation array H 18 The sub-actual numerical range of m 13 ~m 13 For example, the above eighth sub-initial homography transformation array H 18 In this case, 69.8 is the 8th sub-initial homography transformation array H 18 If H is the only initial element with 18 The maximum initial element and the minimum initial element can be H in this example. 18 The sub-actual numerical range is 69.8 to 69.8.

[0209] In this way, the 8th sub-initial homography transformation array H 18If each initial element in needs to be quantized, the sub-quantization coefficient b8 can be (69.8-69.8) / 255=0.0. 18 If each initial element in needs to be quantized, the sub-quantization coefficient b8' can be (69.8-69.8) / 65535=0.0.

[0210] 8th sub-initial homography transformation array H 18 After the sub-quantization coefficients b (for example, b8 and b8') are determined, the eighth sub-initial homography transformation array H 18 can be quantized.

[0211] For example, in the int8 data format, the above 8th sub-initial homography transformation array H 18 is quantized to obtain the 8th subquantized homography transformation array H 28 is obtained as follows:

number

[0212] where H 18min h 18min is an array formed by H 18min All data in h 18min (8th sub-initial homography transformation array H 18 The minimum value of each initial element in the equation is 69.8).

[0213] The above 8th sub-initial homography transformation array H in int16 data format 18 is quantized to obtain the 8th subquantized homography transformation array H 28 is obtained as follows:

number

[0214] In this example, the fourth sub-quantized homography transform array H 24 , the fifth sub-quantized homography transformation array H 25 , the seventh subquantized homography transformation array H 27 and the 8th subquantized homography transformation array H 28 together form the quantized homography transformation matrix H2.

[0215] As can be seen, by dividing the initial homography transformation matrix H1, each divided sub-initial homography transformation array H 1a The accuracy of the quantized homography transformation matrix H2 finally generated by quantizing each of the initial homography transformation matrix H1 is improved, and this accuracy is not lost in the process of quantizing the initial homography transformation matrix H1.

[0216] For example, the fourth sub-quantized homography transformation array H obtained by quantizing in the int8 data format 24 , the fifth sub-quantized homography transformation array H 25 , the seventh subquantized homography transformation array H 27 and the 8th subquantized homography transformation array H 28 are dequantized, and the four dequantized subarrays obtained by the dequantization are column-stitched. The stitched matrix is ​​compared with the initial homography transformation matrix H1 to analyze the accuracy loss.

[0217] Illustratively, the fourth sub-quantized homography transform array H 24 quantized element h in 24a is, h 24a *b4+h 14min Through the calculation of the inverse quantization formula, each inverse quantization element h7 in the inverse quantization array H7 can be obtained, that is, h7=h 24a *b4+h 14min The finally obtained dequantized array H7 is as follows:

number

[0218] Similarly, the fifth subquantized homography transformation array H 25 quantized element h in 25a is, h 25a *b5+h 15min Through the calculation of the inverse quantization formula, each inverse quantization element h8 in the inverse quantization array H8 can be obtained, that is, h8=h 25a *b5+h 15min The finally obtained dequantized array H8 is as follows:

number

[0219] 7th subquantized homography transform array H 27 quantized element h in 27a is, h 27a *b7+h 17min Through the calculation of the inverse quantization formula, each inverse quantization element h9 in the inverse quantization array H9 can be obtained, that is, h9=h 27a *b7+h 17min The finally obtained dequantized array H9 is as follows:

number

[0220] 8th subquantized homography transform array H 28 quantized element h in 28a is, h 28a *b8+h 18 min, the inverse quantization array H 10 Each inverse quantized element h in 10 can be obtained, that is, h 10 =h 28a *b8+h 18 min, and the final inverse quantized array H 10 is as follows:

number

[0221] Next, the inverse quantization array H7, the inverse quantization array H8, the inverse quantization array H9, and the inverse quantization array H 10 is stitched in the reverse process of the division method of the initial homography transformation matrix H1, and the matrix H 11 is obtained as follows:

number

[0222] matrix H 11 By comparing with the initial homography transformation matrix H1, we can see that there is no loss of accuracy.

[0223] 8 is another flow block diagram of generating a coordinate mapping matrix in an image data processing method based on homography transformation according to an exemplary embodiment of the present disclosure. Referring to FIG. 8, in a possible embodiment, a homography transformation constant and a first sub-quantized homography transformation array H 21 Specifically, the step of generating the first intermediate quantized homography transform array based on

[0224] In S2051a, the homography transform constant and the fourth subquantized homography transform array H 24 A third intermediate quantized homography transformation matrix F3 is generated based on

[0225] In some examples, the homography transform constant and the fourth sub-quantized homography transform array H 24 and can be multiplied to generate the third intermediate quantized homography transform matrix F3, and in this example, the number of columns of the homography transform constants is the fourth sub-quantized homography transform array H 24 is equal to the number of rows in

[0226] For example, the fourth subquantized homography transform array H 24has two rows, the number of columns of the homography transformation constant is two, that is, the homography transformation constant is (H*W, 2), which can be the third homography transformation constant C. Next, taking the example where H is 3 and W is 3, the third homography transformation constant C can be expressed as follows:

number

[0227] Illustratively, the fourth sub-quantized homography transform array H 24 When the number of columns of is 2, the third homography transform constant C(H*W,2) and the fourth sub-quantized homography transform array H 24 (2,2) to obtain the third intermediate quantized homography transformation matrix F3(H*W,2).

[0228] Next, the fourth subquantized homography transform array H in the above int8 data format 24 Take (2,2) as an example, the third homography transformation constant C and the fourth subquantized homography transformation array H 24 Specifically, the third intermediate quantized homography transformation matrix F3 obtained by multiplying by (2,2) is as follows:

number

[0229] In S2051b, the third intermediate quantized homography transformation matrix F3 and the fifth sub-quantized homography transformation array H 25 Generate a first intermediate quantized homography transform array based on:

[0230] In some examples, the third intermediate quantized homography transformation matrix F3 and the fifth sub-quantized homography transformation array H 25 and can be added to obtain the first intermediate quantized homography transform array.

[0231] Here, the third intermediate quantization homography transformation matrix F3 and the fifth sub-quantization homography transformation array H 25 When adding the above, the number of rows of the third intermediate quantization homography transformation matrix F3 and the number of rows of the fifth sub-quantization homography transformation array H 25 It is necessary to ensure that the number of rows of the third intermediate quantization homography transformation matrix F3 is equal to the number of rows of the fifth sub-quantization homography transformation array H 25 The number of rows is 3.

[0232] Fifth subquantized homography transform array H 25 If the number of rows of is less than 3, the third intermediate quantization homography transformation matrix F3 and the fifth sub-quantization homography transformation array H 25 Before adding and, the fifth sub-quantized homography transformation array H 25 For example, in this example, the fifth sub-quantized homography transform array H 25 If the number of rows of is 2, the fifth sub-quantized homography transformation array H 25 The fifth subquantized homography transformation array H 25 The quantized element in the last row in can be copied once.

[0233] Similarly, the number of columns of the third intermediate quantization homography transformation matrix F3 and the fifth sub-quantization homography transformation array H 25 For example, if the number of columns of the third intermediate quantization homography transformation matrix F3 is 2, the number of columns of the fifth sub-quantization homography transformation array H 25 The number of columns of the fifth sub-quantized homography transform array H 25 If the number of columns of is 1, the fifth sub-quantized homography transformation array H 25 The fifth subquantized homography transformation array H 25 A column of quantized elements in can be copied once.

[0234] Next, the fifth subquantized homography transformation array H in the int8 data format 25Take (2,1) as an example, the third intermediate quantization homography transformation matrix F3 and the fifth sub-quantization homography transformation array H 25 Specifically, the first intermediate quantized homography transform array F5 obtained by adding (2,1) is as follows:

number

[0235] Accordingly, in some examples, the step of generating the second intermediate quantized homography transform array based on the homography transform constant and the second sub-quantized homography transform array may specifically include the following steps.

[0236] In S2053a, the homography transform constant and the 7th subquantized homography transform array H 27 A fourth intermediate quantized homography transformation matrix F4 is generated based on the above.

[0237] In some examples, a homography transform constant, e.g., a third homography transform constant C and a seventh subquantized homography transform array H 27 and can be multiplied to generate the fourth intermediate quantized homography transform matrix F4. In this example, the number of columns of the homography transform constants is the same as that of the seventh sub-quantized homography transform array H 27 is equal to the number of rows in

[0238] For example, when the number of columns of the homography transformation constants is two, the seventh sub-quantized homography transformation array H 27 The number of rows of the 7th subquantized homography transform array H 27 If the number of rows in is one, the homography transformation constant (e.g., the third homography transformation constant C) and the seventh subquantized homography transformation array H 27 Before multiplying by and, the seventh sub-quantized homography transform array H 27 The seventh subquantized homography transformation array H 27 A quantized element in a row can be copied once.

[0239] Next, the seventh subquantized homography transformation array H of the int8 data format mentioned above 27 Take (1,2) as an example, the third homography transformation constant C(3*3,2) and the seventh subquantized homography transformation array H 27 Specifically, the fourth intermediate quantized homography transformation matrix F4 obtained by multiplying by (1,2) is as follows:

number

[0240] In S2053b, the fourth intermediate quantized homography transformation matrix F4 and the eighth sub-quantized homography transformation array H 28 A second intermediate quantized homography transform array F6 is generated based on

[0241] In some examples, the fourth intermediate quantized homography transform matrix F4 and the eighth sub-quantized homography transform array H 28 and can be added to obtain the second intermediate quantized homography transform array F6.

[0242] Here, the fourth intermediate quantization homography transformation matrix F4 and the eighth sub-quantization homography transformation array H 28 When adding the above, the number of rows of the fourth intermediate quantization homography transformation matrix F4 and the number of rows of the eighth sub-quantization homography transformation array H 28 You need to ensure that the number of rows in the

[0243] For example, the fourth intermediate quantization homography transformation matrix F4 has three rows, and the eighth sub-quantization homography transformation array H 28 If the number of rows of is less than 3, the fourth intermediate quantization homography transformation matrix F4 and the eighth sub-quantization homography transformation array H 28 Before adding and, the 8th sub-quantized homography transformation array H 28 The number of lines can be copied to 3.

[0244] For example, in the above example, the eighth sub-quantized homography transform array H 28 If the number of rows of is 1, the 8th sub-quantized homography transformation array H 28 The 8th subquantized homography transformation array H 28 It can be seen that the quantized elements of one row in the 8th sub-quantized homography transform array H 28 In this example, the quantized elements in each row are all the same.

[0245] Similarly, the number of columns of the fourth intermediate quantization homography transformation matrix F4 and the eighth sub-quantization homography transformation array H 28 For example, if the number of columns of the fourth intermediate quantization homography transformation matrix F4 is 1, the number of columns of the eighth sub-quantization homography transformation array H 28 The number of columns in is also 1.

[0246] Next, the 8th subquantized homography transformation array H of the above int8 data format 28 Take (1,1) as an example, the fourth intermediate quantization homography transformation matrix F4 and the eighth sub-quantization homography transformation array H 28 Specifically, the second intermediate quantized homography transform array F6 obtained by adding (1,1) is as follows:

number

[0247] Next, a coordinate mapping matrix M3 is generated based on the first intermediate quantized homography transform array F5 and the second intermediate quantized homography transform array F6.

[0248] In some possible implementations, the first intermediate quantized homography transform array F5 and the second intermediate quantized homography transform array F6 can be divided to obtain the coordinate mapping matrix M3.

[0249] For example, the coordinate mapping matrix M3 finally obtained by dividing F5 and F6 is as follows:

number

[0250] By comparing M3 and M1 with M0, it is found that M3 is closer to M0, and the initial homography transformation matrix H1 is converted into the fourth sub-initial homography transformation array H 14 , the fifth sub-initial homography transformation array H 15 , the seventh sub-initial homography transformation array H 17 and the 8th sub-initial homography transformation array H 18 and divide it into the fourth sub-initial homography transformation array H 14 , the fifth sub-initial homography transformation array H 15 , the seventh sub-initial homography transformation array H 17 and the 8th sub-initial homography transformation array H 18 By quantizing each of these, a fourth sub-quantized homography transformation array H 24 , the fifth sub-quantized homography transformation array H 25 , the seventh subquantized homography transformation array H 27 and the 8th subquantized homography transformation array H 28 and accordingly generates the fourth sub-quantized homography transformation array H 24 , the fifth sub-quantized homography transformation array H 25 , the seventh subquantized homography transformation array H 27 and the 8th subquantized homography transformation array H 28 The accuracy of the coordinate mapping matrix generated by is also improved, and the accuracy of the target image data at the second viewpoint is improved.

[0251] [Example Device] 9 is a structural diagram of an image data processing device based on homography transformation according to an exemplary embodiment of the present disclosure. Referring to FIG. 9, an embodiment of the present disclosure further provides an image data processing device based on homography transformation, which includes: an acquisition module 200, a quantization module 400, and a generation module 600.

[0252] The acquisition module 200 acquires an initial homography transformation matrix corresponding to the initial image data at a first viewpoint, where each initial element in the initial homography transformation matrix occupies a first number of bytes.

[0253] If H1 represents the initial homography transformation matrix, it is as follows:

number

[0254] where h 11 , h 12 , h 13 , , h 33 are the initial elements in H1. Each initial element in this initial homography transformation matrix H1 is represented in a float32 data format, the number of first bytes is 4, and the number of bits is 32.

[0255] The quantization module 400 quantizes each initial element in the initial homography transform matrix to generate a quantized homography transform matrix, where each quantized element in the quantized homography transform matrix occupies a second number of bytes, and the second number of bytes is smaller than the first number of bytes.

[0256] Specifically, after the initial homography transformation matrix H1 is obtained, each initial element in the initial homography transformation matrix H1 can be quantized into a quantized element occupying a second number of bytes, and this initial homography transformation matrix H1 is finally quantized into a quantized homography transformation matrix H2.

[0257] In some examples, the data format of the quantization element can be int8, that is, each initial element is quantized in the int8 data format to generate a quantized homography transformation matrix H2, and the number of second bytes occupied by each quantization element in this quantized homography transformation matrix H2 is 1, that is, the number of bits occupied by each quantization element is 8.

[0258] In some other examples, the data format of the quantization element can be int16, that is, each initial element is quantized in the data format of int16 to generate a quantized homography transformation matrix H2, and the number of second bytes occupied by each quantized element in this quantized homography transformation matrix H2 is 2, that is, the number of bits occupied by each quantized element is 16.

[0259] The generation module 600 is for generating a coordinate mapping matrix based on the quantized homography transformation matrix.

[0260] For example, an intermediate homography transformation matrix E can be generated based on the quantized homography transformation matrix H2 and the first homography transformation constant A, and then a coordinate mapping matrix M can be generated based on the intermediate homography transformation matrix E.

[0261] Additionally, the generation module 600 further determines target image data at a second viewpoint based on the initial image data and the coordinate mapping matrix.

[0262] For example, after the coordinate mapping matrix M1 is determined, the generation module 600 can remove index values ​​in the input data based on the coordinate mapping matrix M1 to finally obtain output data, and then transform the input data into output data of another viewpoint through the image data processing method based on homography transformation of an embodiment of the present disclosure. As can be understood, the input data can include orientations (e.g., coordinate values ​​in a pixel coordinate system) of multiple sampling points in the initial image data at a first viewpoint and image data at these orientations. For example, referring to Figure 1, the initial image data has four sampling points P, Q, R, and S, and the input data includes coordinate values ​​in the pixel coordinate system of the sampling points P, Q, R, and S and image data corresponding to each point.

[0263] In addition, the output data includes the orientation of the sampling points in the corresponding initial image data in the target image data at the second viewpoint and the target image data. For example, the output data includes the coordinate values ​​of sampling points P, Q, R, and S at the second viewpoint and the image data of each sampling point, thereby determining the target image data at the second viewpoint, that is, the coordinate mapping matrix generated by the homography transformation matrix, which transforms the initial image data at the first viewpoint into the target image data at the second viewpoint.

[0264] As can be seen from the above, in the image data processing device based on homography transformation of the embodiment of the present disclosure, the quantization module 400 quantizes each initial element occupying a first number of bytes in the initial homography transformation matrix into a quantized element occupying a second number of bytes, i.e., quantizes the initial homography transformation matrix in which each initial element occupies the first number of bytes into a quantized homography transformation matrix in which the quantized elements are smaller than the first number of bytes, thereby reducing the occupied space of each quantized element in the quantized homography transformation matrix; then, the generation module 600 generates a coordinate mapping matrix using the quantized homography transformation matrix; and generates target image data at the second viewpoint using this coordinate mapping matrix and the initial image data at the first viewpoint.

[0265] Compared to the related art, which generates a coordinate mapping matrix using a homography transformation matrix formed with initial elements occupying a first number of bytes (e.g., 4, corresponding data format is float32), the embodiment of the present disclosure quantizes the initial homography transformation matrix occupying a first number of bytes to generate a quantized homography transformation matrix, in which the quantized elements occupy a second number of bytes, which is smaller than the first number of bytes, and then uses the quantized homography matrix with a relatively small number of bytes after quantization to generate a coordinate mapping matrix. Then, the target image data at the second viewpoint is determined based on the initial image data and the coordinate mapping matrix, thereby on the one hand saving the space occupied by the quantized homography transformation matrix and on the other hand improving the speed of generating the coordinate mapping matrix and improving the overall efficiency of the image data processing process based on this homography transformation.

[0266] 10 is another structural diagram of an image data processing device based on homography transformation according to an exemplary embodiment of the present disclosure. Referring to FIG. 10, the image data processing device based on homography transformation according to an embodiment of the present disclosure may further include a determination module 300.

[0267] The determination module 300 determines the quantization coefficient based on the second byte number.

[0268] Specifically, the determination module 300 may include a first determination unit 310, a second determination unit 320, and a third determination unit 330. Here, the first determination unit 310 determines a quantization display range according to the second byte number.

[0269] The second determining unit 320 determines the actual value range according to the maximum initial element and the minimum initial element in the initial homography transformation matrix.

[0270] The third determining unit 330 determines the quantization coefficient according to the quantization display range and the actual value range.

[0271] After the quantization coefficients are determined, the quantization module 400 can quantize each initial element in the initial homography transform matrix based on the quantization coefficients to generate a quantized homography transform matrix.

[0272] 11 is a structural diagram of a quantization module in an image data processing apparatus based on homography transformation according to an exemplary embodiment of the present disclosure. Referring to FIG. 11, in some examples, a quantization module 400 may include a division unit 410, a quantization unit 420, and a generation unit 430.

[0273] The division unit 410 divides the initial homography transformation matrix to generate a plurality of sub-initial homography transformation arrays.

[0274] For example, the division unit 410 can divide the initial homography transformation matrix along the column direction of the initial homography transformation matrix to generate at least two sub-initial homography transformation arrays, where the at least two sub-initial homography transformation arrays include a first sub-initial homography transformation array and a second sub-initial homography transformation array.

[0275] The quantization unit 420 quantizes each initial element in each sub-initial homography transform array to generate a corresponding sub-quantized homography transform array.

[0276] For example, the quantization unit 420 may include a first quantization subunit and a second quantization subunit.

[0277] Here, the first quantization subunit quantizes the first sub-initial homography transform array to generate a first sub-quantized homography transform array, and the second quantization subunit quantizes the second sub-initial homography transform array to generate a second sub-quantized homography transform array.

[0278] The generation unit 430 generates a quantized homography transform matrix based on the plurality of sub-quantized homography transform matrices.

[0279] For example, the generating unit 430 may include a first generating subunit, a second generating subunit, and a third generating subunit.

[0280] Here, the first generation subunit generates a first intermediate quantized homography transform array based on the homography transform constants and the first sub-quantized homography transform array, the second generation subunit generates a second intermediate quantized homography transform array based on the homography transform constants and the second sub-quantized homography transform array, and the third generation subunit generates a coordinate mapping matrix based on the first intermediate quantized homography transform array and the second intermediate quantized homography transform array.

[0281] In some other examples, the dividing unit 410 may include a first dividing subunit and a second dividing subunit.

[0282] Here, the first dividing subunit can divide the first sub-initial homography transformation array along the row direction of the first sub-initial homography transformation array to generate a plurality of third sub-initial homography transformation arrays, and the plurality of third sub-initial homography transformation arrays include a fourth sub-initial homography transformation array and a fifth sub-initial homography transformation array.

[0283] The second division subunit can divide the second sub-initial homography transformation array along the row direction of the second sub-initial homography transformation array to generate a plurality of sixth sub-initial homography transformation arrays, and the plurality of sixth sub-initial homography transformation arrays include a seventh sub-initial homography transformation array and an eighth sub-initial homography transformation array.

[0284] In this example, the first quantization subunit may quantize the fourth sub-initial homography transform array to generate a fourth quantized homography transform array. The first quantization subunit may further quantize the fifth sub-initial homography transform array to generate a fifth quantized homography transform array.

[0285] Here, the first generation subunit can generate a third intermediate quantized homography transform array based on the homography transform constant and the fourth sub-quantized homography transform array, and can further generate a first intermediate quantized homography transform array based on the third intermediate quantized homography transform array and the fifth sub-quantized homography transform array.

[0286] In addition, the second quantization subunit can quantize the seventh sub-initial homography transform array to generate a seventh quantized homography transform array, and can further quantize the eighth sub-initial homography transform array to generate an eighth quantized homography transform array.

[0287] Here, the second generation subunit can generate a fourth intermediate quantized homography transform array based on the homography transform constant and the seventh sub-quantized homography transform array, and further generate a second intermediate quantized homography transform array based on the fourth intermediate quantized homography transform array and the eighth sub-quantized homography transform array.

[0288] [Example electronic devices] An electronic device according to an embodiment of the present disclosure will be described below with reference to Fig. 12. The electronic device may be either one or both of the first device and the second device, or may be a standalone device independent of them, which is capable of communicating with the first device and the second device and receiving input signals collected therefrom. As will be understood, in the embodiment of the present disclosure, the input signal is input data.

[0289] FIG. 12 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure.

[0290] As shown in FIG. 12, the electronic device 11 includes one or more processors 111 and a memory 112.

[0291] The processor 111 may be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and may control other components in the electronic device 11 to perform desired functions.

[0292] The memory 112 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, and flash memory. The computer-readable storage medium may store one or more computer program instructions, and the processor 111 may execute the one or more computer program instructions to realize the image data processing method based on the homography transformation of each embodiment of the present disclosure and / or other desired functions. The computer-readable storage medium may also store various contents, such as an input signal, a signal component, and a noise component.

[0293] By way of example, electronic device 11 may further include input devices 113 and output devices 114 connected to one another via a bus system and / or other form of connection (not shown).

[0294] For example, when the electronic device 11 is a first device or a second device, the input device 113 may include, but is not limited to, a keyboard, a mouse, a microphone, or a microphone array for acquiring input data (i.e., an input signal) to be subjected to homography transformation. When the electronic device 11 is standalone, the input device 113 may be a communication network connector for receiving input signals collected from the first device and the second device.

[0295] The output device 114 can output various information including determined output data to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0296] 11 shows only some of the components related to the present disclosure in the electronic device 11, and omits components such as a bus and an input / output interface. In addition, the electronic device 11 may include any appropriate components depending on the application.

[0297] Exemplary Computer Program Products and Computer-Readable Storage Media In addition to the above methods and apparatus, embodiments of the present disclosure provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform steps in the image data processing method based on homography transformation according to various embodiments of the present disclosure described herein above in the "Exemplary Method" section.

[0298] The computer program product may have program code for carrying out operations of embodiments of the present disclosure written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and traditional procedural programming languages ​​such as "C" or similar programming languages. The program code may execute entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on the remote computing device or a server.

[0299] Additionally, embodiments of the present disclosure further provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in the image data processing method based on homography transformation according to various embodiments of the present disclosure described herein above in the "Exemplary Method" section.

[0300] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0301] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the benefits, advantages, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and these benefits, advantages, effects, etc. are not necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details of the above disclosure are merely illustrative and easy-to-understand functions and are not limiting, and the above details do not necessarily limit the present disclosure to those realized by the above specific details.

[0302] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. 1. An image data processing method based on homography transformation, comprising: obtaining an initial homography transformation matrix corresponding to the initial image data at a first viewpoint, wherein each initial element in the initial homography transformation matrix occupies a first number of bytes; quantizing each of the initial elements in the initial homography transform matrix to generate a quantized homography transform matrix, wherein each quantized element in the quantized homography transform matrix occupies a second number of bytes, the second number of bytes being smaller than the first number of bytes; generating a coordinate mapping matrix based on the quantized homography transformation matrix; and obtaining target image data at a second viewpoint based on the initial image data and the coordinate mapping matrix.

2. prior to the step of quantizing each of the initial elements in the initial homography transformation matrix to generate a quantized homography transformation matrix, The method further includes determining a quantization coefficient based on the second number of bytes; quantizing each of the initial elements in the initial homography transform matrix to generate a quantized homography transform matrix; The method of claim 1 , further comprising quantizing each of the initial elements in the initial homography transform matrix based on the quantization coefficients to generate the quantized homography transform matrix.

3. determining a quantization coefficient based on the second number of bytes, determining a quantization display range based on the second number of bytes; determining an actual value range according to a maximum initial element and a minimum initial element in the initial homography transformation matrix; determining the quantization coefficient based on the quantization display range and the real-world value range.

4. quantizing each of the initial elements in the initial homography transform matrix to generate a quantized homography transform matrix, generating a plurality of sub-initial homography transformation arrays based on the initial homography transformation matrix; quantizing each initial element in each of the sub-initial homography transform arrays to generate a corresponding sub-quantized homography transform array; and generating the quantized homography transformation matrix based on a plurality of the sub-quantized homography transformation matrices.

5. Before the step of quantizing each initial element in each of the sub-initial homography transform arrays to generate a corresponding sub-quantized homography transform array, The method comprises: determining a quantization display range based on the second byte number; determining a sub-actual numerical range of each of the sub-initial homography transform arrays based on a maximum initial element and a minimum initial element in each of the sub-initial homography transform arrays; determining sub-quantized coefficients of each of the sub-initial homography transform arrays based on the quantized representation range and the sub-actual value range; further comprising quantizing each initial element in each of the sub-initial homography transform arrays to generate a corresponding sub-quantized homography transform array; 5. The method of claim 4, further comprising quantizing each initial element in each of the sub-initial homography transform arrays based on the sub-quantized coefficients to generate a corresponding each of the sub-quantized homography transform arrays.

6. The step of generating a plurality of sub initial homography transformation arrays based on the initial homography transformation matrix includes: dividing the initial homography transformation matrix along a column direction of the initial homography transformation matrix to generate at least two sub-initial homography transformation arrays, the at least two sub-initial homography transformation arrays including a first sub-initial homography transformation array and a second sub-initial homography transformation array; quantizing each initial element in each of the sub-initial homography transform arrays to generate a corresponding sub-quantized homography transform array; quantizing the first sub-initial homography transform array to generate a first sub-quantized homography transform array; 5. The method of claim 4, further comprising: quantizing the second sub-initial homography transform array to generate a second sub-quantized homography transform array.

7. The step of generating a coordinate mapping matrix based on the quantized homography transformation matrix includes: generating a first intermediate quantized homography transform array based on a homography transform constant and the first sub-quantized homography transform array; generating a second intermediate quantized homography transform array based on the homography transform constants and the second sub-quantized homography transform array; generating the coordinate mapping matrix based on the first intermediate quantized homography transform array and the second intermediate quantized homography transform array.

8. Before the step of quantizing the first sub-initial homography transform array to generate a first sub-quantized homography transform array, The method further includes dividing the first sub-initial homography transform array along a row direction of the first sub-initial homography transform array to generate a plurality of third sub-initial homography transform arrays, the plurality of third sub-initial homography transform arrays including a fourth sub-initial homography transform array and a fifth sub-initial homography transform array; quantizing the first sub-initial homography transform array to generate a first sub-quantized homography transform array, quantizing the fourth sub-initial homography transform array to generate a fourth quantized homography transform array; 8. The method of claim 7, further comprising: quantizing the fifth sub-initial homography transform array to generate a fifth quantized homography transform array.

9. generating a first intermediate quantized homography transform array based on a homography transform constant and the first sub-quantized homography transform array, generating a third intermediate quantized homography transform array based on a homography transform constant and the fourth sub-quantized homography transform array; generating the first intermediate quantized homography transform array based on the third intermediate quantized homography transform array and the fifth sub-quantized homography transform array.

10. before the step of quantizing the second sub-initial homography transform array to generate a second sub-quantized homography transform array; the method includes dividing the second sub-initial homography transform array along a row direction of the second sub-initial homography transform array to generate a plurality of sixth sub-initial homography transform arrays, the plurality of sixth sub-initial homography transform arrays including a seventh sub-initial homography transform array and an eighth sub-initial homography transform array; quantizing the second sub-initial homography transform array to generate a second sub-quantized homography transform array, quantizing the seventh sub-initial homography transform array to generate a seventh quantized homography transform array; 8. The method of claim 7, further comprising: quantizing the eighth sub-initial homography transform array to generate an eighth quantized homography transform array.

11. generating a second intermediate quantized homography transform array based on a homography transform constant and the second sub-quantized homography transform array, generating a fourth intermediate quantized homography transform array based on a homography transform constant and the seventh sub-quantized homography transform array; and generating a second intermediate quantized homography transform array based on the fourth intermediate quantized homography transform array and the eighth sub-quantized homography transform array.

12. An image data processing apparatus based on homography transformation, comprising: an acquisition module for acquiring an initial homography transformation matrix corresponding to the initial image data at a first viewpoint, wherein each initial element in the initial homography transformation matrix occupies a first number of bytes; a quantization module for quantizing each initial element in the initial homography transform matrix to generate a quantized homography transform matrix, wherein each quantized element in the quantized homography transform matrix occupies a second number of bytes, the second number of bytes being smaller than the first number of bytes; a generation module for generating a coordinate mapping matrix based on the quantized homography transformation matrix, and for obtaining target image data at a second viewpoint based on the initial image data and the coordinate mapping matrix.

13. A computer-readable storage medium on which a computer program is stored, 12. A computer-readable storage medium, comprising: a computer program for executing the image data processing method based on homography transformation according to any one of claims 1 to 11.

14. 1. An electronic device including a processor and a memory for storing instructions executable by the processor, 12. An electronic device, characterized in that the processor reads the executable instructions from the memory and executes them to realize the image data processing method based on homography transformation according to any one of claims 1 to 11.

Images