Projection image geometric correction method and system based on dynamic physical constraint

Abstract

The invention provides a projection image geometric correction method and system based on dynamic physical constraints, and relates to the technical field of projection image processing. According to the method, the minimum enclosing rectangle of the trapezoidal projection area is determined through physical constraint modeling, only pixels in the area are processed, and the calculation amount is reduced; expanding the original image by adopting a one-pixel mirror image, ensuring that adjacent points required by interpolation are all in the expanded image, and eliminating boundary holes and sawteeth; the source image coordinates and the interpolation weight are pre-calculated and stored through the LUT, and floating point operation is avoided in combination with integer real-time interpolation processing. Real-time processing larger than or equal to 30 fps can be achieved on an embedded platform, and the projection image quality and the processing efficiency are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of projection image processing, and in particular to a projection image geometric correction method and system based on dynamic physical constraints. Background Art

[0002] In projection display technology, due to the possible change in the relative position between the projection device and the projection surface, the projected image often suffers from geometric distortion, which affects the display effect. Existing projection correction solutions have many defects. For example, the traditional projection transformation follows the basic homography model and requires real-time calculation of 9 degrees of freedom parameters, resulting in O(n²) complexity, which cannot meet the requirements of vehicle-mounted systems. 25fps real-time requirement; Bicubic interpolation has high accuracy (PSNR>40dB) but large computational complexity, including O(kn²) (k≥16), such as Figure 3 As shown, nearest neighbor interpolation is highly efficient but prone to aliasing (PSNR < 30dB). Furthermore, existing technologies suffer from significant resource consumption: for example, during the pre-processing of projection material, a large amount of storage space is required to ensure that the original scene is not lost after the relative spatial position of the projection device is transformed. The problem of a missing physical model: existing solutions ignore key physical constraints. The actual projection area is a dynamic trapezoidal shape, not a rectangle, resulting in redundant calculations across the entire frame.

[0003] Based on this, existing technologies also have problems such as hollow effect, boundary distortion, huge resource consumption and lack of physical models. Summary of the Invention

[0004] The purpose of the present invention is to overcome at least one technical problem existing in the prior art and to provide a method and system for geometric correction of projected images based on dynamic physical constraints.

[0005] On the one hand, an embodiment of the present invention provides a method for geometric correction of projected images based on dynamic physical constraints, the method comprising: step S1, acquiring the posture data of the projection device in real time through a vehicle body sensor; step S2, establishing a dynamic trapezoidal domain of the projection area according to the posture data; step S3, calculating the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area; step S4, performing boundary extension processing on the original source image to obtain an extended source image; step S5, obtaining an inverse perspective transformation matrix according to discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the extended source image; step S6, decomposing the corresponding mapping coordinates in the extended source image into an integer part and a decimal part; step S7, quantizing the decimal part into In the Q8 format, the integer part and the decimal part are packed and compressed into an N-byte data structure, and stored in order according to the position of each pixel in the target image area to form a LUT table; step S8, calling the LUT table to read the LUT entry corresponding to the target pixel in the target image area, and obtaining the corresponding integer part and decimal part; step S9, locating the coordinates of the four neighboring pixels of the target pixel in the extended source image according to the integer part; step S10, calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm of pure integer operations based on the coordinates of the four neighboring pixels and the decimal part; step S11, repeating steps S8-S10 by traversing all target pixels in the target image area, and outputting the corrected image.

[0006] Furthermore, the step S3, calculating the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area, includes: step S301, mapping the original source image projection area of ​​the rectangle to the dynamic trapezoidal area in the actual scene using homography change; step S302, calculating the coordinates of the four vertices of the trapezoidal area; step S303, obtaining the minimum and maximum values ​​of all vertex coordinates in the X-axis direction and the Y-axis direction; step S304, obtaining the target image area Rt based on the minimum and maximum values: wherein, .

[0007] Furthermore, the step S4, performing boundary extension processing on the original source image to obtain the extended source image, includes: extending one pixel around each of the four sides of the original source image, and filling the pixel values ​​of the extended part in a mirror image manner to form the extended source image. .

[0008] Furthermore, the step S5, obtaining an inverse perspective transformation matrix according to the discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the extended source image includes: step S501, obtaining an inverse perspective transformation matrix according to the discretization of the projection device parameter space Step S502: traverse each target pixel in the target image area Rt , according to the inverse perspective transformation matrix Convert it to original source image coordinates ; Step S503, the original source image coordinates The coordinate point is obtained by normalizing the homogeneous coordinates ( , ); Step S504, the original source image coordinates The coordinate reference system is translated from the original source image to the extended source image to obtain the corresponding mapping coordinates in the extended source image .

[0009] Furthermore, the step S6 of decomposing the corresponding mapping coordinates in the extended source image into integer parts and decimal parts includes: step S601, using the floor function to Decompose into integer parts ,in , ; Step S602, Decompose into decimal parts ,in , .

[0010] Furthermore, the step S7, quantizing the fractional part into Q8 format, packing and compressing the integer part and the fractional part into an N-byte data structure, and storing them sequentially according to the position of each pixel in the target image area, to form a LUT table includes: step S701, converting the fractional part into Q8 format by using the Round function and Quantized into 8-bit integers and ,in, , ; Step S702, 、 、 and Packed and compressed into a 32-bit data structure, it uses a 9-bit x-coordinate and a 7-bit y-coordinate bit field design, combined with an 8-bit Q8 weight, to form a 4-byte or pixel LUT storage structure.

[0011] Furthermore, the step S9, locating the coordinates of the four neighboring pixels of the target pixel in the extended source image according to the integer part, includes: step S901, for each target pixel coordinate in the target image area Rt, , read the corresponding 4-byte data from the LUT table, including: 、 、 and ; Step S902, according to , Locate the extended source image The 4 neighboring pixels in include: ; .

[0012] Furthermore, the step S10, calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm of pure integer operation based on the coordinates of the four neighboring pixel points and the decimal part, includes: step S1001, calculating the brightness or color intensity value of the target pixel in the corrected image based on the bilinear interpolation algorithm of pure integer operation based on the decimal part. and Calculate the four coefficients of the bilinear interpolation, including: ; ; Step S1002, read the pixel values ​​of the four neighboring pixels, multiply them with W0, W1, W2 and W3 respectively, and then sum them to obtain a total weighted value; Step S1003, normalize the total weighted value through 16-bit shift to obtain the pixel value of the target pixel after real-time interpolation processing.

[0013] Furthermore, in step S503, the original source image coordinates The coordinate point is obtained by normalizing the homogeneous coordinates ( , ) and then also include: and Clamp the image using the Clamp function to prevent it from exceeding the boundaries of the extended source image. These include: ; ;in, The size data of the original source image.

[0014] In the second aspect, an embodiment of the present invention provides a projection image geometric correction system based on dynamic physical constraints, which is implemented using the above-mentioned projection image geometric correction method based on dynamic physical constraints. The system includes a real-time processing system and a pre-calculation system; the pre-calculation system integrates a homography parameter space discretization unit, a LUT generation engine and a LUT compression storage unit, and the real-time processing system integrates an image input unit, a boundary expansion unit, a physical constraint processor, a LUT index engine, an integer difference unit and a correction image output unit; the image input unit is used to receive the input original source image; the physical constraint processor is used to obtain the posture data of the projection device in real time through the vehicle body sensor, establish a dynamic trapezoidal domain of the projection area according to the posture data, and calculate the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area; the boundary expansion unit is used to perform boundary expansion processing on the original source image to obtain an extended source image; the homography parameter space discretization unit is used to obtain an inverse perspective transformation matrix based on the discretization of the projection device parameter space The LUT generation engine is configured to traverse each pixel in the target image area according to the inverse perspective transformation matrix, calculate the corresponding mapping coordinates in the extended source image, decompose the corresponding mapping coordinates in the extended source image into an integer part for representing the coordinates of the upper left reference pixel in the original source image and a decimal part for representing the offset from the upper left reference pixel, and quantize the decimal part into a Q8 format; the LUT compression storage unit is configured to pack and compress the integer part and the decimal part into an N-byte data structure, and store them sequentially according to the position of each pixel in the target image area to form a LUT table; the LUT indexing engine is configured to call the LUT table to read the LUT entry corresponding to the target pixel in the target image area, obtain the corresponding integer part and decimal part, and locate the coordinates of four neighboring pixels in the extended source image according to the integer part; the integer difference unit is configured to calculate the brightness or color intensity value of the target pixel in the corrected image based on the four neighboring pixel coordinates and the decimal part using a bilinear interpolation algorithm using pure integer operations; and the corrected image output unit is configured to output the corrected image.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-mentioned method for geometric correction of projected images based on dynamic physical constraints.

[0016] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the above-mentioned method for geometric correction of projected images based on dynamic physical constraints.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention determines the minimum circumscribed rectangle of the trapezoidal projection area through physical constraint modeling and processes only the pixels within the area, which greatly reduces the amount of calculation and improves processing efficiency.

[0018] (2) The original image is expanded by 1-pixel mirroring, which ensures that the neighboring points required for interpolation are all within the expanded image, effectively eliminating boundary holes and aliasing, and improving the quality of the projected image.

[0019] (3) By pre-calculating and storing the source image coordinates and interpolation weights through LUT, combined with integer real-time interpolation processing, floating-point operations are avoided, allowing the system to achieve real-time processing of ≥30fps on embedded platforms, meeting the real-time requirements of vehicle-mounted systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below with reference to the accompanying drawings and examples.

[0021] Figure 1 This is a flow chart of a method for geometric correction of projected images based on dynamic physical constraints provided in Example 1 of the present invention.

[0022] Figure 2 This is a schematic diagram of an original source image provided by Example 1 of the present invention.

[0023] Figure 3 This is a schematic diagram of a corrected image obtained by using the existing technology provided by the present invention.

[0024] Figure 4 This is a schematic diagram of an intermediate image of a corrected image formed by inverse transformation processing during the correction process provided by Example 1 of the present invention.

[0025] Figure 5 This is a schematic diagram of a corrected image formed using the method of this embodiment provided by Example 1 of the present invention.

[0026] Figure 6 This is a schematic diagram of the structure of a projection image geometric correction system based on dynamic physical constraints provided in Example 2 of the present invention.

[0027] Figure 7 This is a partial block diagram of an electronic device provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0028] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations as sequential processes, many of the operations therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0029] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and a similar second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0030] The present invention will now be described in detail with reference to the accompanying drawings. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore only shows the components related to the present invention.

[0031] Example 1 For ease of understanding, the overall inventive concept of the present invention is explained here: The present invention aims to provide a method and system for geometric correction of projected images based on dynamic physical constraints to solve the problems of projected image distortion and low processing efficiency in the prior art. The overall inventive concept of the present invention is to solve the problems of boundary distortion, lack of real-time performance, and high resource consumption in traditional projected image geometric correction through the collaborative design of "pre-calculation + physical constraint optimization + full integer real-time processing", and to achieve efficient, high-quality real-time correction of projected images on embedded platforms. Specifically, it can be broken down into the following core ideas: 1. Physically constrained cropping eliminates invalid calculations: Given the dynamic trapezoidal shape of the actual projection area, the minimum bounding rectangle of the trapezoid is determined using homography changes. Only pixels within this rectangle are processed, reducing invalid calculations at the source and resolving the full-frame redundant calculation issue in traditional solutions.

[0032] 2. Boundary extension eliminates distortion: The original image is expanded by 1-pixel mirroring to ensure that the neighboring points required for interpolation fall within the expanded image range, avoiding holes and jagged edges caused by missing boundary pixels. At the same time, it eliminates complex boundary judgment branches and improves processing efficiency.

[0033] 3. Separation of pre-calculation and real-time processing: A LUT (lookup table) pre-calculation mechanism is designed to generate and store source image coordinates and interpolation weights in advance (using a compact 4-byte / pixel structure). LUT data is directly called during real-time processing to avoid delays caused by dynamic calculation and meet real-time requirements.

[0034] 4. Integerized architecture avoids floating-point operations: The decimal part in the interpolation calculation is quantized into Q8 format, and interpolation fusion and normalization are completed through pure integer multiplication and addition operations and bit shift operations, avoiding the high resource consumption of floating-point operations and adapting to the hardware limitations of embedded platforms.

[0035] In summary, the present invention forms a complete real-time geometric correction solution for projected images through physical constraint modeling to optimize the calculation range, boundary expansion to ensure image quality, pre-calculation to reduce real-time burden, and integer operation to improve efficiency. It achieves a balance between real-time processing of ≥30fps and high-quality correction effect on embedded platforms.

[0036] The specific implementation is as follows: like Figure 1 As shown, it is a flow chart of a projection image geometric correction method based on dynamic physical constraints provided by the present invention.

[0037] As an example, the method includes: step S1, obtaining the posture data of the projection device in real time through the vehicle body sensor; step S2, establishing a dynamic trapezoidal domain of the projection area according to the posture data; step S3, calculating the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area; step S4, performing boundary expansion processing on the original source image to obtain an extended source image; step S5, obtaining an inverse perspective transformation matrix according to the discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the extended source image; step S6, decomposing the corresponding mapping coordinates in the extended source image into an integer part and a decimal part; step S7, quantizing the decimal part into Q8 format, and converting the integer part and the decimal part into a binary matrix. The parts are packed and compressed into an N-byte data structure, and stored in order according to the position of each pixel in the target image area to form a LUT table; step S8, calling the LUT table to read the LUT entry corresponding to the target pixel in the target image area, and obtaining the corresponding integer part and decimal part; step S9, locating the coordinates of the four neighboring pixels of the target pixel in the extended source image according to the integer part; step S10, using a bilinear interpolation algorithm of pure integer operation to calculate the brightness or color intensity value of the target pixel in the corrected image based on the coordinates of the four neighboring pixels and the decimal part; step S11, repeating steps S8-S10 by traversing all target pixels in the target image area, and outputting the corrected image.

[0038] In some feasible implementations, the step S3 of calculating the minimum bounding rectangle of the dynamic trapezoidal domain as the target image area includes: step S301, using homography to transform the original source image of the rectangle (such as Figure 2 The projection area is mapped to a dynamic trapezoidal area in the actual scene; step S302, calculating the coordinates of the four vertices of the trapezoidal area; step S303, obtaining the minimum and maximum values ​​of all vertex coordinates in the X-axis direction and the Y-axis direction; step S304, obtaining the target image area Rt based on the minimum and maximum values: wherein, , is the minimum value of all vertices in the X-axis direction, is the maximum value of all vertices in the X-axis direction, is the minimum value of all vertices in the Y-axis direction, It is the maximum value of all vertices in the Y-axis direction.

[0039] Specifically, if a vehicle HUD system needs to project navigation information (a 320×80 pixel image) onto the front windshield, the projection area is actually a dynamic trapezoid (not a rectangle) due to the dynamic change in the relative position of the windshield and the projection device during driving. The coordinates of its four vertices are: (20,10), (290,15), (280,70), and (30,65). The minimum enclosing rectangle (Rt) is calculated based on the coordinates of the trapezoid vertices to obtain: =20, =290, =10, =70, the effective pixel area is 270×60 (only 63% of the original 320×80 image), reducing the invalid pixel calculation by 37%.

[0040] Preferably, a homography matrix H is applied in the homography transformation, which describes the point mapping relationship between two planes (such as the original source image plane and the projection target plane) and is represented by a 3×3 matrix. The parameters of the homography matrix H are obtained in the following ways: Step 1: Collect corresponding point pairs In the original source image (rectangular area) and the actual projection area (dynamic trapezoid), manually or automatically mark at least 4 pairs of non-collinear corresponding feature points (such as corner points). For example, the 4 vertices (( ),( ),( ),( )), and the four vertices of the projected trapezoid ( , ),( , ),( , ),( , Step 2: Solve the homography matrix H. Using corresponding point pairs, the nine parameters of the homography matrix H are solved using the least squares method. Because the projection area is a dynamic trapezoid (for example, in an in-vehicle scenario, the projection surface changes with the vehicle's posture), H must be updated in real time using physical sensors. This includes: using on-board sensors (such as cameras, gyroscopes, and distance sensors) to collect dynamic parameters such as the projection surface's position and angle in real time; and dynamically adjusting the homography matrix H based on a preset physical model.

[0041] Preferably, based on a preset physical model, dynamically adjusting the homography matrix H includes: The physical model is constructed by the relative posture parameters of the projection device and the target plane, specifically including the pitch angle of the projection device ( ), yaw angle ( )、Roll Angle( ) and projection distance (d). The above parameters are collected in real time by the vehicle body sensor to establish the dynamic projection plane equation , and then the analytical expression of the homography matrix H is derived as: ; Where K is the internal parameter matrix of the projection device, R is the rotation matrix, which is determined by the pitch angle , yaw angle , roll angle Calculated, n is the unit normal vector of the projection plane, through It is deduced that t is the translation vector from the projection device to the projection plane, d is the projection distance, and I is Based on this, the physical parameters are mapped to the 9 degrees of freedom of the homography matrix to achieve dynamic adjustment.

[0042] in, , , .

[0043] In some feasible implementations, the step S4 of performing boundary extension processing on the original source image to obtain the extended source image includes: extending one pixel around each side of the original source image, and filling the pixel values ​​of the extended part in a mirror image manner to form the extended source image. .

[0044] Preferably, since in the prior art, when correcting the image boundary, the four source image neighboring points required to calculate the target point P are It may fall outside the boundaries of the original source image, resulting in incorrect interpolation (holes) or the use of incorrect values ​​(aliasing). Traditional methods require complex boundary condition judgments, increase the computational burden and have poor results. Therefore, this embodiment performs a simple 1-pixel boundary mirroring expansion on the original source image to form an extended source image. That is, copy a circle of adjacent pixels around the original image. Ensure that for any target point P in the corrected image (in Rt), the four neighboring points required for interpolation Must all fall into the extended source image In this way, the boundary judgment branch can be completely eliminated, and a branchless and complete bilinear interpolation can be achieved, fundamentally solving the boundary hole and aliasing problems.

[0045] Specifically, for example, a vehicle HUD system needs to project navigation information (a 320×80 pixel image) onto the windshield. Pixel mirroring expands the original 320×80 navigation image: the left edge is expanded by one column, with the pixel values ​​being the same as the original left edge (column x=0); the right edge is expanded by one column, with the pixel values ​​being the same as the original right edge (column x=319); and the top and bottom edges are expanded by one row each, with the pixel values ​​being the same as the original top edge (row y=0) and bottom edge (row y=79), respectively. The resulting image size is 322×82 pixels.

[0046] In some feasible implementations, the step S5, obtaining an inverse perspective transformation matrix based on the discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the extended source image, includes: step S501, obtaining an inverse perspective transformation matrix based on the discretization of the projection device parameter space Step S502: traverse each target pixel in the target image area Rt , according to the inverse perspective transformation matrix Convert it to original source image coordinates ; Step S503, the original source image coordinates The coordinate point is obtained by normalizing the homogeneous coordinates ( , ); Step S504, the original source image coordinates The coordinate reference system is translated from the original source image to the extended source image to obtain the corresponding mapping coordinates in the extended source image .

[0047] Preferably, in step S501, the inverse perspective transformation matrix obtained by discretizing the projection device parameter space is Including: Projection equipment parameter space discretization refers to the physical parameters (pitch angle , yaw angle , roll angle and projection distance d) are uniformly sampled within the value range: where the pitch angle by is the step length; yaw angle by is the step length; projection distance The step size is 0.05m. The homography matrix H is calculated for each sampling point through the above preset physical model, and the inverse matrix of the homography matrix is ​​obtained to obtain the inverse perspective transformation matrix = .

[0048] Preferably, the inverse perspective transformation matrix is the inverse matrix of the homography matrix H. Since the solution of the homography matrix H has been explained in detail above, its inverse matrix will not be explained here. To elaborate.

[0049] In some feasible implementations, the step S6 of decomposing the corresponding mapping coordinates in the extended source image into an integer part and a decimal part includes: step S601, using the floor function to Decompose into integer parts ,in , ; Step S602, Decompose into decimal parts ,in , .

[0050] Preferably, ; Integer part, value range ; ; Integer part, value range ; ;Decimal part, range [0,1); ; decimal part, range [0,1).

[0051] Specifically, use calculate ,like The coordinates are (100, 50), and the corresponding coordinates in the original source image are (91.41, 54.12), which are mapped to the extended source image coordinates. , ; Separate the integer part , and decimal part , .

[0052] In some feasible implementations, step S7, quantizing the fractional part into a Q8 format, packing and compressing the integer part and the fractional part into an N-byte data structure, and storing them sequentially according to the position of each pixel in the target image area to form a LUT table includes: Step S701: Round the decimal part and Quantized into 8-bit integers and ,in, , ; Step S702, 、 、 and Packed and compressed into a 32-bit data structure, it uses a 9-bit x-coordinate and a 7-bit y-coordinate bit field design, combined with an 8-bit Q8 weight, to form a 4-byte or pixel LUT storage structure.

[0053] Preferably, the decimal part and Convert from floating point to fixed point (Q8 format: multiply by 256 and convert to an integer between 0 and 255, which is equivalent to retaining 8 decimal places). Bit field compression includes: (9 bits are enough to represent the X coordinate range of the expanded image), (7 bits are enough to represent the Y coordinate range of the expanded image), (8 bits), (8 bits) are packed and compressed into a 32-bit (4-byte) data structure. Structure diagram: .

[0054] Specifically, 6. 0.3 as an example, the decimal part is quantized into Q8 format ( ),( ); Press "9-digit X integer + 7-digit Y integer + 8-digit +8 bits " is stored in a bit field structure, with each pixel occupying 4 bytes (such as Expressed in 9 bits, Represented by 7 bits). The total size of the LUT is 16200×4 bytes = 64.8KB, which can be directly stored in the RAM of the embedded platform. It should be noted that the LUT table is not generated in real time, but is generated offline or online before real-time correction. The pre-generated LUT table can be directly called in the subsequent real-time correction process. That is, only when the projection device or vehicle posture changes significantly (such as when the vehicle starts or the bumps reach the threshold), it is necessary to re-run the process of pre-calculating the LUT table and generate a new LUT table. That is, the separation architecture of pre-calculation and real-time processing is adopted in this embodiment, which can effectively reduce the amount of calculation. Among them, when the bumps reach the threshold, it includes: the pitch angle or yaw angle changes greater than 5°, or the distance changes greater than 10cm. When the vehicle body sensor detects that the posture change exceeds the above threshold, the LUT table re-precalculation process is triggered.

[0055] It should be noted that due to and The maximum rounding error is less than 0.4%, and the effect on the interpolation result can be ignored. Therefore, in practical applications, It can avoid 8-bit integer overflow, that is, avoid Overflow.

[0056] In some feasible implementations, the step S9, locating the coordinates of the four neighboring pixels of the target pixel in the extended source image according to the integer part, includes: step S901, for each target pixel coordinate in the target image area Rt, , read the corresponding 4-byte data from the LUT table, including: 、 、 and ; Step S902, according to , Locate the extended source image The 4 neighboring pixels in include: ; ; ; .

[0057] Specifically, assuming the original source image has integer coordinates (2,3), determine the four source image neighboring points required for bilinear interpolation (all within the expanded image): P00: the top-left neighbor, coordinates (2,3), with a pixel value of 100; P01: the top-right neighbor, coordinates (3,3), with a pixel value of 200; P10: the bottom-left neighbor, coordinates (2,4), with a pixel value of 150; and P11: the bottom-right neighbor, coordinates (3,4), with a pixel value of 250. Because of the 1-pixel mirroring extension, even if the base coordinates are close to the edge of the original image, the neighboring points will not be out of range. For example, if the base coordinates are (0,0), the expanded neighboring points (-1,0) are still valid.

[0058] Based on this, combined Figure 3 、 Figure 4 and Figure 5 As shown, it can be seen that compared with the prior art, this embodiment effectively solves the problems of jagged edges and holes in the projected image by performing 1-pixel mirror expansion and inverse transformation.

[0059] In some feasible implementations, the step S10, calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm of pure integer operations based on the coordinates of the four neighboring pixel points and the decimal part, includes: step S1001, calculating the brightness or color intensity value of the target pixel in the corrected image based on the coordinates of the four neighboring pixel points and the decimal part using a bilinear interpolation algorithm of pure integer operations. and Calculate the four coefficients of the bilinear interpolation, including: ; ; Step S1002, read the pixel values ​​of the four neighboring pixels, multiply them with W0, W1, W2 and W3 respectively, and then sum them to obtain a total weighted value; Step S1003, normalize the total weighted value through 16-bit shift to obtain the pixel value of the target pixel after real-time interpolation processing.

[0060] Specifically, assuming , after quantization , , after quantization , calculate the 4 coefficients of bilinear interpolation: ; ; Multiply the 4 neighboring pixel values ​​by the corresponding coefficients and add them up to get: ; Because the coefficient is magnified when calculating =65536 times, the final result needs to be shifted right 16 bits (equivalent to dividing by 65536), that is That is, the pixel value of the target pixel after real-time interpolation is 150. The entire process is completed only through integer operations, without floating-point calculations and boundary judgments, and because it is based on the extended image, holes and jagged edges are avoided.

[0061] In some feasible implementations, in step S503, the original source image coordinates The coordinate point is obtained by normalizing the homogeneous coordinates ( , ) and then also include: and Clamp the image using the Clamp function to prevent it from exceeding the boundaries of the extended source image. These include: ; ;in, The dimensions of the original source image are clamped to the calculated coordinates to ensure they do not exceed the boundaries of the expanded image. (Although theoretically, this should not happen after expansion, the numerical calculation may have slight errors.)

[0062] In some feasible implementations, in order to intuitively demonstrate the advantages of this embodiment, a comparative experiment was conducted with the prior art: Processed on S32K344 with a main frequency of 160MHz The results are shown in Table 1: Table 1:

[0063] The above implementation uses physical constraint modeling to determine the minimum bounding rectangle of the trapezoidal projection area, processing only the pixels within that area to reduce computational complexity. A 1-pixel mirror image is used to expand the original image, ensuring that all neighboring points required for interpolation are within the expanded image, eliminating boundary holes and jagged edges. A LUT (Low-Position Table) pre-calculates and stores the source image coordinates and interpolation weights, combining integer-based real-time interpolation processing to avoid floating-point operations. This invention can achieve real-time processing at ≥30 fps on embedded platforms, effectively improving projected image quality and processing efficiency.

[0064] Example 2 See also Figure 6 , this embodiment provides a structural schematic diagram of a projection image geometric correction system based on dynamic physical constraints.

[0065] As an example, the system is implemented using the projection image geometric correction method based on dynamic physical constraints described in Example 1. The system includes a real-time processing system 1 and a pre-calculation system 2.

[0066] The pre-calculation system 2 integrates a homography parameter space discretization unit 20, a LUT generation engine 21 and a LUT compression storage unit 22, and the real-time processing system 1 integrates an image input unit 10, a boundary expansion unit 11, a physical constraint processor 12, a LUT index engine 13, an integer difference unit 14 and a corrected image output unit 15.

[0067] The image input unit 10 is used to receive an input original source image.

[0068] The physical constraint processor 12 is used to obtain the posture data of the projection device in real time through the vehicle body sensor, establish a dynamic trapezoidal domain of the projection area according to the posture data, and calculate the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area.

[0069] The boundary extension unit 11 is used to perform boundary extension processing on the original source image to obtain an extended source image.

[0070] The homography parameter space discretization unit 20 is used to obtain the inverse perspective transformation matrix according to the discretization of the projection device parameter space. .

[0071] The LUT generation engine 21 is used to traverse each pixel in the target image area according to the inverse perspective transformation matrix, calculate its corresponding mapping coordinates in the extended source image, decompose the corresponding mapping coordinates in the extended source image into an integer part for representing the coordinates of the upper left corner reference pixel in the original source image and a decimal part for representing the offset from the upper left corner reference pixel, and quantize the decimal part into Q8 format.

[0072] The LUT compression storage unit 22 is used to pack and compress the integer part and the fractional part into an N-byte data structure, and store them sequentially according to the position of each pixel in the target image area to form a LUT table.

[0073] The LUT index engine 13 is used to call the LUT table to read the LUT entry corresponding to the target pixel in the target image area, obtain the corresponding integer part and decimal part, and locate the coordinates of the four neighboring pixels in the extended source image according to the integer part.

[0074] The integer difference unit 14 is used to calculate the brightness or color intensity value of the target pixel in the corrected image based on the coordinates of the four neighboring pixel points and the decimal part using a bilinear interpolation algorithm of pure integer operations.

[0075] The corrected image output unit 15 is used to output the corrected image.

[0076] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0077] It is worth noting that all modules involved in this embodiment are logical units. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, to highlight the innovations of the present invention, this embodiment does not include units that are not closely related to solving the technical problems proposed by the present invention. However, this does not mean that other units do not exist in this embodiment.

[0078] Example 3 See also Figure 7 An embodiment of the present invention further provides an electronic device, comprising: a memory and a processor; the memory stores at least one program instruction; the processor implements the projection image geometric correction method based on dynamic physical constraints provided in Example 1 by loading and executing the at least one program instruction.

[0079] The memory 702 and processor 701 are connected using a bus. The bus can include any number of interconnected buses and bridges, connecting various circuits of one or more processors 701 and memory 702. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and, therefore, are not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor 701 is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor 701.

[0080] The processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 702 can be used to store data used by the processor 701 when performing operations.

[0081] Example 4 An embodiment of the present invention further provides a storage medium storing a method for geometric correction of projected images based on dynamic physical constraints. When executed by a processor, this program implements the steps of the method for geometric correction of projected images based on dynamic physical constraints as described above. Because this storage medium incorporates all the technical solutions of all the aforementioned embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of these embodiments, and therefore will not be further elaborated upon here.

[0082] The above is only an embodiment of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the field are aware of all common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A method for geometric correction of projected images based on dynamic physical constraints, characterized in that: The method comprises: Step S1: obtaining the position data of the projection device in real time through the vehicle body sensor; Step S2: establishing a dynamic trapezoidal domain of the projection area according to the posture data; Step S3, calculating the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area; Step S4: performing boundary extension processing on the original source image to obtain an extended source image; Step S5: obtaining an inverse perspective transformation matrix according to the discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the extended source image; Step S6: decomposing the corresponding mapping coordinates in the extended source image into an integer part and a decimal part; Step S7, quantizing the fractional part into a Q8 format, packing and compressing the integer part and the fractional part into an N-byte data structure, and storing them sequentially according to the position of each pixel in the target image area to form a LUT table; Step S8: calling the LUT table to read the LUT entry corresponding to the target pixel in the target image area, and obtaining the corresponding integer part and decimal part; Step S9: locating the coordinates of four neighboring pixels of the target pixel in the extended source image according to the integer part; Step S10, calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm of pure integer operations based on the coordinates of the four neighboring pixel points and the decimal part; Step S11, repeatedly executing steps S8-S10 by traversing all target pixels in the target image area, and outputting a corrected image.

2. The method for geometric correction of projected images based on dynamic physical constraints according to claim 1, characterized in that: The step S3 of calculating the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area includes: Step S301: Mapping the rectangular original source image projection area to a dynamic trapezoidal area in the actual scene using homography change; Step S302: Calculate the coordinates of the four vertices of the trapezoidal area; Step S303: Obtain the minimum and maximum values ​​of all vertex coordinates in the X-axis direction and the Y-axis direction; Step S304: Obtain the target image region Rt based on the minimum value and the maximum value: in, .

3. The method for geometric correction of projected images based on dynamic physical constraints according to claim 1, characterized in that: The step S4, performing boundary extension processing on the original source image to obtain the extended source image, includes: extending one pixel around each of the four sides of the original source image, and filling the pixel values ​​of the extended part in a mirror image manner to form the extended source image. .

4. The method for geometric correction of projected images based on dynamic physical constraints according to claim 1, characterized in that: The step S5 of obtaining an inverse perspective transformation matrix according to the discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the extended source image includes: Step S501: Obtain an inverse perspective transformation matrix based on the discretization of the projection device parameter space. ; Step S502: traverse each target pixel in the target image area Rt , according to the inverse perspective transformation matrix Convert it to original source image coordinates ; Step S503: coordinate the original source image The coordinate point is obtained by normalizing the homogeneous coordinates ( , ); Step S504: The original source image coordinates The coordinate reference system is translated from the original source image to the extended source image to obtain the corresponding mapping coordinates in the extended source image .

5. The method for geometric correction of projected images based on dynamic physical constraints according to claim 4, characterized in that: The step S6 of decomposing the corresponding mapping coordinates in the extended source image into an integer part and a decimal part comprises: Step S601, use the floor function to Decompose into integer parts ,in , ; Step S602: Decompose into decimal parts ,in , .

6. The method for geometric correction of projected images based on dynamic physical constraints according to claim 5, characterized in that: The step S7, quantizing the fractional part into a Q8 format, packing and compressing the integer part and the fractional part into an N-byte data structure, and storing them sequentially according to the position of each pixel in the target image area to form a LUT table, includes: Step S701: Round the decimal part and quantized into 8-bit integers to obtain and ,in, , ; Step S702: 、 、 and Packed and compressed into a 32-bit data structure, it uses a 9-bit x-coordinate and a 7-bit y-coordinate bit field design, combined with an 8-bit Q8 weight, to form a 4-byte or pixel LUT storage structure.

7. The method for geometric correction of projected images based on dynamic physical constraints according to claim 6, characterized in that: The step S9 of locating the coordinates of four neighboring pixels of the target pixel in the extended source image according to the integer part includes: Step S901: For each target pixel coordinate in the target image area Rt, , read the corresponding 4-byte data from the LUT table, including: 、 、 and ; Step S902: , Locate the extended source image The 4 neighboring pixels in include: ； ； ； 。 8. The method for geometric correction of projected images based on dynamic physical constraints according to claim 7, characterized in that: The step S10 of calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm of pure integer operations based on the coordinates of the four neighboring pixel points and the decimal part includes: Step S1001: Based on the and Calculate the four coefficients of the bilinear interpolation, including: ； ； Step S1002: Read the pixel values ​​of the four neighboring pixels, multiply them by W0, W1, W2 and W3 respectively, and then sum them to obtain a total weighted value; Step S1003 : normalizing the total weighted value by 16-bit shifting to obtain the pixel value of the target pixel after real-time interpolation processing.

9. The method for geometric correction of projected images based on dynamic physical constraints according to claim 4, characterized in that: In step S503, the original source image coordinates The coordinate point is obtained by normalizing the homogeneous coordinates ( , ) and then also include: Regarding the and Clamp the image using the Clamp function to prevent it from exceeding the boundaries of the extended source image. These include: ； ； in, The size data of the original source image.

10. A system for geometric correction of projected images based on dynamic physical constraints, implemented using the method for geometric correction of projected images based on dynamic physical constraints according to any one of claims 1 to 9, characterized in that: The system includes a real-time processing system and a pre-calculation system; The pre-calculation system integrates a homography parameter space discretization unit, a LUT generation engine, and a LUT compression storage unit, and the real-time processing system integrates an image input unit, a boundary expansion unit, a physical constraint processor, a LUT index engine, an integer difference unit, and a corrected image output unit; The image input unit is used to receive an input original source image; The physical constraint processor is used to obtain the posture data of the projection device in real time through the vehicle body sensor, establish a dynamic trapezoidal domain of the projection area according to the posture data, and calculate the minimum circumscribed rectangle of the dynamic trapezoidal domain as the target image area; The boundary extension unit is used to perform boundary extension processing on the original source image to obtain an extended source image; The homography parameter space discretization unit is used to obtain the inverse perspective transformation matrix according to the discretization of the projection device parameter space. ; The LUT generation engine is configured to traverse each pixel in the target image area according to the inverse perspective transformation matrix, calculate the corresponding mapping coordinates of the pixel in the extended source image, decompose the corresponding mapping coordinates in the extended source image into an integer part representing the coordinates of the upper left corner reference pixel in the original source image and a fractional part representing the offset from the upper left corner reference pixel, and quantize the fractional part into a Q8 format; The LUT compression storage unit is used to pack and compress the integer part and the fractional part into an N-byte data structure, and store them in order according to the position of each pixel in the target image area to form a LUT table; The LUT indexing engine is configured to call the LUT table to read the LUT entry corresponding to the target pixel in the target image area, obtain the corresponding integer part and decimal part, and locate the coordinates of four neighboring pixels in the extended source image according to the integer part; The integer difference unit is used to calculate the brightness or color intensity value of the target pixel in the corrected image based on the coordinates of the four neighboring pixel points and the decimal part using a bilinear interpolation algorithm of pure integer operations; The corrected image output unit is used to output the corrected image.

Images