A method and system for geometric correction of projected images based on dynamic physical constraints

Abstract

This invention provides a method and system for geometric correction of projected images based on dynamic physical constraints, belonging to the field of projection image processing technology. The method determines the minimum bounding rectangle of the trapezoidal projection region through physical constraint modeling, processing only pixels within this region to reduce computational load; it uses a 1-pixel mirror to expand the original image, ensuring that all neighboring points required for interpolation are within the expanded image, eliminating boundary holes and jagged edges; and it pre-computes and stores the source image coordinates and interpolation weights through a LUT, combined with integer-based real-time interpolation processing, avoiding floating-point operations. This invention can achieve real-time processing of ≥30fps on embedded platforms, effectively improving the quality and processing efficiency of projected images.

Description

Technical Field

[0001] This invention relates to the field of projection image processing technology, and specifically to a method and system for geometric correction of projection images based on dynamic physical constraints. Background Technology

[0002] In projection display technology, geometric distortion often occurs in the projected image due to the potential changes in the relative position between the projection device and the projection surface, affecting the display effect. Existing projection correction schemes have many shortcomings. For example, traditional projection transformation follows the basic homography model, requiring real-time calculation of 9 degrees of freedom parameters, resulting in O(n²) complexity, which cannot meet the requirements of in-vehicle systems. The real-time requirement is 25fps; bicubic interpolation offers high accuracy (PSNR>40dB) but involves significant computational complexity, specifically O(kn²) (k≥16). Figure 3 As shown, nearest neighbor interpolation is efficient but prone to jagged edges (PSNR < 30dB); furthermore, existing technologies suffer from significant resource consumption: for example, during the preprocessing of projection materials, a large amount of space is required to store the original materials to ensure that the original scene is not lost after the relative spatial position of the projection device changes. There is also the problem of missing physical models: existing solutions ignore key physical constraints, and the actual projection area is a dynamic trapezoidal non-rectangular shape, leading to redundant calculations across the entire frame.

[0003] As a result, existing technologies suffer from problems such as voiding, boundary distortion, huge resource consumption, and lack of physical models. Summary of the Invention

[0004] The purpose of this invention is to overcome at least one technical problem existing in the prior art and to provide a method and system for geometric correction of projected images based on dynamic physical constraints.

[0005] On one hand, embodiments of the present invention provide a method for geometric correction of projected images based on dynamic physical constraints. The method includes: step S1, acquiring pose data of the projection device in real time through vehicle body sensors; step S2, establishing a dynamic trapezoidal domain of the projection area based on the pose data; step S3, calculating the minimum bounding rectangle of the dynamic trapezoidal domain as the target image area; step S4, performing boundary expansion processing on the original source image to obtain an expanded source image; step S5, obtaining an inverse perspective transformation matrix based on the discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the expanded source image; step S6, decomposing the corresponding mapping coordinates in the expanded source image into an integer part and a fractional part; step S7, quantizing the fractional part into... The Q8 format packages and compresses the integer and fractional parts into an N-byte data structure, storing them sequentially according to the pixel position in the target image region to form a LUT table; Step S8: Call the LUT table to read the LUT entry corresponding to the target pixel in the target image region and obtain the corresponding integer and fractional parts; Step S9: Locate the coordinates of the four neighboring pixels of the target pixel in the extended source image based on the integer part; Step S10: Calculate the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm with pure integer operations based on the coordinates of the four neighboring pixels and the fractional part; Step S11: Repeat steps S8-S10 by traversing all target pixels in the target image region to output the corrected image.

[0006] Further, step S3, calculating the minimum bounding rectangle of the dynamic trapezoidal domain as the target image region, includes: step S301, using homography transformation to map the original source image projection region of the rectangle onto the dynamic trapezoidal region in the actual scene; step S302, calculating the coordinates of the four vertices of the trapezoidal region; step S303, obtaining the minimum and maximum values ​​of all vertex coordinates in the X-axis and Y-axis directions; step S304, obtaining the target image region Rt based on the minimum and maximum values: where, .

[0007] Furthermore, step S4, performing boundary expansion processing on the original source image to obtain an expanded source image, includes: expanding the original source image by one pixel on each side, with the pixel values ​​of the expanded portion filled using a mirror method to form the expanded source image. .

[0008] Further, step S5, obtaining the inverse perspective transformation matrix based on the spatial discretization of the projection device parameters, and traversing each pixel in the target image region to calculate its corresponding mapping coordinates in the extended source image, includes: step S501, obtaining the inverse perspective transformation matrix based on the spatial discretization of the projection device parameters. Step S502: Traverse each target pixel within the target image region Rt. According to the inverse perspective transformation matrix Convert it to the original source image coordinates Step S503: The coordinates of the original source image... The coordinates are obtained by homogeneous coordinate normalization. , Step S504: Set the coordinates of the original source image. The coordinate reference system is translated from the original source image to the extended source image to obtain its corresponding mapped coordinates in the extended source image. .

[0009] Furthermore, step S6, decomposing the corresponding mapped coordinates in the extended source image into integer and fractional parts, includes: step S601, using the floor function to... Decompose into integer parts ,in , Step S602, to Decompose into decimal parts ,in , .

[0010] Further, step S7, quantizing the fractional part into Q8 format, and packing and compressing the integer and fractional parts into an N-byte data structure, storing it sequentially according to the pixel position in the target image region to form a LUT table, includes: step S701, using the Round function to quantize the fractional part... and Quantized into 8-bit integers respectively and ,in, , Step S702, , , and The data is packaged and compressed into a 32-bit data structure, using a 9-bit x-coordinate and a 7-bit y-coordinate bit field design, combined with an 8-bit Q8 weight, to form a 4-byte or pixel LUT storage structure.

[0011] Furthermore, step S9, locating the coordinates of the target pixel in the extended source image based on the integer part, includes: step S901, for each target pixel coordinate within the target image region Rt... Read the corresponding 4 bytes of data from the LUT table, including: , , and Step S902, according to , Locate extended source image The four neighboring pixels include: ; .

[0012] Further, step S10, calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm with pure integer operations based on the coordinates of the four neighboring pixels and the fractional part, includes: step S1001, based on the... and The four coefficients of the bilinear interpolation are calculated, including: ; Step S1002: Read the pixel values ​​of the four neighboring pixels, multiply them by W0, W1, W2 and W3 respectively, and sum them to obtain the total weighted value; Step S1003: Normalize the total weighted value by 16 bit shifting to obtain the pixel value of the target pixel after real-time interpolation.

[0013] Furthermore, in step S503, the coordinates of the original source image are... The coordinates are obtained by homogeneous coordinate normalization. , Following this is also: [the following is a list of items / items] and Clamping is performed using the Clamp function to prevent the image from exceeding the boundaries of the extended source image, including: ; ;in, This refers to the dimensions of the original source image.

[0014] Secondly, embodiments of the present invention provide a projection image geometric correction system based on dynamic physical constraints, implemented using the aforementioned projection image geometric correction method based on dynamic physical constraints. The system includes a real-time processing system and a pre-computation system. The pre-computation system integrates a homography parameter space discretization unit, a LUT generation engine, and a LUT compression and storage unit. The real-time processing system integrates an image input unit, a boundary expansion unit, a physical constraint processor, a LUT indexing engine, an integer difference unit, and a corrected image output unit. The image input unit receives the input original source image. The physical constraint processor acquires the pose data of the projection device in real time using vehicle body sensors, establishes a dynamic trapezoidal domain of the projection area based on the pose data, and calculates the minimum bounding rectangle of the dynamic trapezoidal domain as the target image area. The boundary expansion unit performs boundary expansion processing on the original source image to obtain an expanded source image. The homography parameter space discretization unit is used to obtain the inverse perspective transformation matrix based on the projection device parameter space discretization. The LUT generation engine is used to traverse each pixel in the target image region according to the inverse perspective transformation matrix, calculate its corresponding mapping coordinates in the extended source image, decompose the corresponding mapping coordinates in the extended source image into an integer part representing the coordinates of the upper left reference pixel in the original source image and a fractional part representing the offset from the upper left reference pixel, and quantize the fractional part into Q8 format; the LUT compression storage unit is used to pack and compress the integer part and the fractional part into an N-byte data structure, store them sequentially according to the position of each pixel in the target image region, and form a LUT table; the LUT indexing engine is used to call the LUT table to read the LUT entry corresponding to the target pixel in the target image region, obtain the corresponding integer part and fractional part, and locate the coordinates of the four neighboring pixels in the extended source image according to the integer part; the integer difference unit is used to calculate the brightness or color intensity value of the target pixel in the corrected image based on the coordinates of the four neighboring pixels and the fractional part using a bilinear interpolation algorithm with pure integer operations; the corrected image output unit is used to output the corrected image.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described method for geometric correction of projected images based on dynamic physical constraints.

[0016] Fourthly, embodiments of the present invention also provide a readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the above-described method for geometric correction of projected images based on dynamic physical constraints.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] (1) The present invention uses physical constraint modeling to determine the minimum bounding rectangle of the trapezoidal projection area, and only processes the pixels in this area, which greatly reduces the amount of calculation and improves the processing efficiency.

[0019] (2) The original image is extended by 1 pixel mirroring, which ensures that the neighboring points required for interpolation are all within the extended image, effectively eliminating boundary holes and jagged edges, and improving the quality of the projected image.

[0020] (3) By pre-calculating the source image coordinates and interpolation weights through LUT, and combining them with integer real-time interpolation processing, floating-point operations are avoided, enabling the system to achieve real-time processing of ≥30fps on the embedded platform, which meets the real-time requirements of vehicle systems and other systems. Attached Figure Description

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] Figure 1 This is a schematic diagram of the process of a projection image geometric correction method based on dynamic physical constraints provided in Embodiment 1 of the present invention.

[0023] Figure 2 This is a schematic diagram of an original source image provided in Embodiment 1 of the present invention.

[0024] Figure 3 This is a schematic diagram of a corrected image obtained using existing technology, provided by the present invention.

[0025] Figure 4 This is a schematic diagram of an intermediate image of a corrected image formed by inverse transformation during the correction process, as provided in Embodiment 1 of the present invention.

[0026] Figure 5 This is a schematic diagram of a corrected image formed using the method described in Embodiment 1 of the present invention.

[0027] Figure 6 This is a schematic diagram of a projection image geometric correction system based on dynamic physical constraints provided in Embodiment 2 of the present invention.

[0028] Figure 7 This is a partial block diagram of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0029] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0030] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and a similar second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated items listed.

[0031] The present invention will now be described in detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0032] Example 1

[0033] For ease of understanding, the overall inventive concept of this invention is described below: This invention aims to provide a method and system for geometric correction of projected images based on dynamic physical constraints, to solve problems such as projection image distortion and low processing efficiency in existing technologies. The overall inventive concept of this invention solves the problems of boundary distortion, insufficient real-time performance, and high resource consumption in traditional projection image geometric correction through a collaborative design of "pre-computation + physical constraint optimization + fully integer real-time processing," achieving efficient and high-quality real-time correction of projection images on embedded platforms. Specifically, it can be broken down into the following core ideas:

[0034] 1. Physical constraint clipping of invalid calculations: Taking advantage of the dynamic trapezoidal nature of the actual projection area, the minimum bounding rectangle of the trapezoid is determined by homography changes. Only the pixels within this rectangle are processed, reducing the amount of invalid calculations at the source and solving the problem of full-frame redundant calculations in traditional solutions.

[0035] 2. Boundary expansion to eliminate distortion: The original image is expanded by 1 pixel mirroring to ensure that all neighboring points required for interpolation fall within the range of the expanded image, avoiding holes and jagged edges caused by missing boundary pixels, while eliminating complex boundary judgment branches and improving processing efficiency.

[0036] 3. Separation of pre-computation and real-time processing: A LUT (Look-Up Table) pre-computation mechanism is designed to generate and store the source image coordinates and interpolation weights in advance (using a compact structure of 4 bytes / pixel). During real-time processing, the LUT data is directly called to avoid the delay caused by dynamic calculation and meet the real-time requirements.

[0037] 4. Integer-based architecture avoids floating-point operations: The fractional part in the interpolation calculation is quantized into Q8 format, and interpolation fusion and normalization are completed through pure integer multiplication and addition operations and bit shift operations, avoiding the high resource consumption of floating-point operations and adapting to the hardware limitations of embedded platforms.

[0038] In summary, this invention optimizes the computational range through physical constraint modeling, ensures image quality through boundary expansion, reduces real-time burden through pre-computation, and improves efficiency through integer-based operations, forming a complete real-time geometric correction scheme for projected images. This achieves a balance between real-time processing at ≥30fps and high-quality correction effects on embedded platforms.

[0039] The specific implementation method is as follows:

[0040] like Figure 1 The diagram shown is a flowchart of a projection image geometric correction method based on dynamic physical constraints provided by the present invention.

[0041] As an example, the method includes: Step S1, acquiring the pose data of the projection device in real time through the vehicle body sensor; Step S2, establishing a dynamic trapezoidal domain of the projection area based on the pose data; Step S3, calculating the minimum bounding rectangle of the dynamic trapezoidal domain as the target image area; Step S4, performing boundary expansion processing on the original source image to obtain an expanded source image; Step S5, obtaining the inverse perspective transformation matrix based on the discretization of the projection device parameter space, traversing each pixel in the target image area, and calculating its corresponding mapping coordinates in the expanded source image; Step S6, decomposing the corresponding mapping coordinates in the expanded source image into an integer part and a fractional part; Step S7, quantizing the fractional part into Q8 format, and decomposing the integer part and the fractional part into... Partially packaged and compressed into an N-byte data structure, stored sequentially according to the position of each pixel in the target image region, forming a LUT table; Step S8: Call the LUT table to read the LUT entry corresponding to the target pixel in the target image region, and obtain the corresponding integer part and fractional part; Step S9: Locate the coordinates of the four neighboring pixels of the target pixel in the extended source image based on the integer part; Step S10: Calculate the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm with pure integer operations based on the coordinates of the four neighboring pixels and the fractional part; Step S11: Repeat steps S8-S10 by traversing all target pixels in the target image region, and output the corrected image.

[0042] In some feasible implementations, step S3, calculating the minimum bounding rectangle of the dynamic trapezoidal domain as the target image region, includes: step S301, utilizing homography transformation to transform the original source image of the rectangle (e.g., Figure 2 (As shown) The projection area is mapped to a dynamic trapezoidal area in the actual scene; Step S302: Calculate the coordinates of the four vertices of the trapezoidal area; Step S303: Obtain the minimum and maximum values ​​of all vertex coordinates in the X-axis and Y-axis directions; Step S304: Obtain the target image area Rt based on the minimum and maximum values: where, , The minimum value of all vertices along the X-axis. This represents the maximum value of all vertices along the X-axis. The minimum value of all vertices along the Y-axis. This represents the maximum value of all vertices along the Y-axis.

[0043] Specifically, if a vehicle HUD system needs to project navigation information (320×80 pixel image) onto the windshield, the projection area is actually a dynamic trapezoid (not a rectangle) due to the dynamic changes in the relative position of the windshield and the projection device while the vehicle is moving. The coordinates of its four vertices are (20,10), (290,15), (280,70), and (30,65). The minimum bounding rectangle (Rt) is calculated based on the vertex coordinates of the trapezoid. =20, =290, =10, =70, the effective pixel area is 270×60 (only 63% of the original 320×80 image), reducing invalid pixel calculation by 37%.

[0044] Preferably, the homography transformation utilizes a homography matrix H, which describes the point mapping relationship between two planes (such as the original source image plane and the projected target plane), represented by a 3×3 matrix. The parameters of the homography matrix H are obtained in the following ways:

[0045] Step 1: Collect corresponding point pairs. In the original source image (rectangular region) and the actual projected region (dynamic trapezoid), manually or automatically mark at least 4 pairs of non-collinear corresponding feature points (such as corner points). For example, the 4 vertices of the original source image (( ),( ),( ),( )), and the four vertices of the projected trapezoid ( , ),( , ),( , ),( , Step 2: Solve for the homography matrix H. Using corresponding point pairs, solve for the nine parameters of the homography matrix H using the least squares method. Since the projection area is a dynamic trapezoid (such as the projection surface changing with the vehicle's posture in a vehicle scene), H needs to be updated in real time using physical sensors, including: real-time acquisition of dynamic parameters such as the position and angle of the projection surface through vehicle sensors (such as cameras, gyroscopes, and distance sensors); and dynamic adjustment of the homography matrix H based on a preset physical model.

[0046] Preferably, dynamically adjusting the homography matrix H based on a preset physical model includes:

[0047] The physical model is constructed using the relative pose parameters between the projection device and the target plane, specifically including the pitch angle of the projection device (…). ), yaw angle ( ), roll angle ( The dynamic projection plane equation is established based on the parameters collected in real time by the vehicle body sensors, including the projection distance (d) and the projection distance (d). Therefore, the analytical expression for the homography matrix H is derived as follows:

[0048] ;

[0049] In the formula, K is the intrinsic parameter matrix of the projection device, and R is the rotation matrix, which is determined by the pitch angle. Yaw angle Roll angle The calculation shows that n is the unit normal vector of the projection plane. The derivation yields that t is the translation vector from the projection device to the projection plane, d is the projection distance, and I is... The identity matrix. Based on this, physical parameters are mapped to the nine degrees of freedom of the homography matrix, enabling dynamic adjustment.

[0050] in, , , .

[0051] In some feasible implementations, step S4, performing boundary expansion processing on the original source image to obtain an expanded source image, includes: expanding the original source image by one pixel on each side, with the pixel values ​​of the expanded portion filled using a mirror image method to form the expanded source image. .

[0052] Preferably, in the prior art, when correcting image boundaries, the calculation of the target point P requires four source image neighboring points. The image may fall outside the boundaries of the original source image, leading to incorrect interpolation (holes) or the use of incorrect values ​​(aliasing). Traditional methods require complex boundary condition judgments, increasing computational burden and yielding poor results. Therefore, this embodiment performs a simple 1-pixel boundary mirroring extension of the original source image to form an extended source image. This involves copying a ring of adjacent pixels around the original image. This ensures that for any target point P in the corrected image (within Rt), the four neighboring points required for interpolation are... They must all fall within the extended source image. This completely eliminates boundary judgment branches, achieving branchless, complete bilinear interpolation and fundamentally solving the problems of boundary holes and jagged edges.

[0053] Specifically, taking a vehicle HUD system that needs to project navigation information (320×80 pixel image) onto the windshield as an example, pixel mirroring expansion of the original 320×80 navigation image involves expanding its boundaries: the left boundary is expanded by 1 column, with pixel values ​​identical to the original left edge (x=0 column); the right boundary is expanded by 1 column, with pixel values ​​identical to the original right edge (x=319 column); the top and bottom boundaries are each expanded by 1 row, with pixel values ​​identical to the original top edge (y=0 row) and bottom edge (y=79 row), respectively. The expanded image size is 322×82 pixels.

[0054] In some feasible implementations, step S5, obtaining the inverse perspective transformation matrix based on the discretization of the projection device parameter space, and traversing each pixel in the target image region to calculate its corresponding mapping coordinates in the extended source image, includes: step S501, obtaining the inverse perspective transformation matrix based on the discretization of the projection device parameter space. Step S502: Traverse each target pixel within the target image region Rt. According to the inverse perspective transformation matrix Convert it to the original source image coordinates Step S503: The coordinates of the original source image... The coordinates are obtained by homogeneous coordinate normalization. , Step S504: Set the coordinates of the original source image. The coordinate reference system is translated from the original source image to the extended source image to obtain its corresponding mapped coordinates in the extended source image. .

[0055] Preferably, in step S501, the inverse perspective transformation matrix is ​​obtained by discretizing the projection device parameter space. This includes: spatial discretization of projection device parameters, referring to the discretization of physical parameters (tilt angle). Yaw angle Roll angle And the projection distance d) are uniformly sampled within the range of values: where, pitch angle by Step size; yaw angle by Step size; projected distance Using a step size of 0.05m, the homography matrix H is calculated for each sampling point using the aforementioned preset physical model. The inverse of the homography matrix is ​​then used to obtain the inverse perspective transformation matrix. = .

[0056] Preferably, the inverse perspective transformation matrix Let be the inverse matrix of the homography matrix H. Since the solution method for the homography matrix H has already been explained in detail above, its inverse matrix will not be discussed here. To elaborate further.

[0057] In some feasible implementations, step S6, decomposing the corresponding mapped coordinates in the extended source image into integer and fractional parts, includes: step S601, using the floor function to... Decompose into integer parts ,in , Step S602, to Decompose into decimal parts ,in , .

[0058] Preferred, Integer part, range of values ; ;

[0059] Integer part, range of values ; The decimal part, ranging from [0,1). ; The decimal part, ranging from [0,1).

[0060] Specifically, use calculate ,like The coordinates are (100, 50), which correspond to (91.41, 54.12) in the original source image. Mapping these coordinates to the extended source image gives... , Separate the integer part , and decimal part , .

[0061] In some feasible implementations, step S7, quantizing the fractional part into Q8 format, and packing and compressing the integer and fractional parts into an N-byte data structure, storing it sequentially according to the pixel position in the target image region to form a LUT table, includes:

[0062] Step S701: Use the Round function to round the decimal part. and Quantized into 8-bit integers respectively and ,in, , Step S702, will , , and The data is packaged and compressed into a 32-bit data structure, using a 9-bit x-coordinate and a 7-bit y-coordinate bit field design, combined with an 8-bit Q8 weight, to form a 4-byte or pixel LUT storage structure.

[0063] Preferably, the decimal part and Convert from floating-point to fixed-point (Q8 format: multiply by 256 to convert to an integer between 0 and 255, equivalent to retaining 8 decimal places). Bit-field compression includes: (9 bits are sufficient to represent the X coordinate range of the expanded image) (7 bits are sufficient to represent the Y-coordinate range of the expanded image) (8 bits) (8 bits) are compressed into a single 32-bit (4-byte) data structure. Structure illustration: .

[0064] Specifically, with 6, Taking 0.3 as an example, the decimal part is quantized into Q8 format ( ), ( ); (indicated by "9-bit X integer + 7-bit Y integer + 8-bit") +8 bits The storage is a bit-field structure, with each pixel occupying 4 bytes (e.g., ...). Represented by 9 bits (Represented using 7 bits). The total size of the LUT is 16200 × 4 bytes = 64.8KB, which can be directly stored in the RAM of the embedded platform. It should be noted that the LUT table is not generated in real time, but is generated offline or online before real-time correction. The pre-generated LUT table can be directly called in the subsequent real-time correction process. That is, the process of re-running this pre-calculated LUT table and generating a new LUT table is only required when there is a significant change in the attitude of the projection device or vehicle (such as vehicle start-up or when the bump reaches a threshold). In other words, this embodiment adopts a pre-calculation and real-time processing separation architecture, which can effectively reduce the amount of computation. Among them, the bump reaching the threshold includes: pitch angle or yaw angle change greater than 5°, or distance change greater than 10cm. When the vehicle body sensor detects that the attitude change exceeds the above threshold, the LUT table re-pre-calculation process is triggered.

[0065] It should be noted that, due to and The maximum error of rounding is less than 0.4%, and its impact on the interpolation result is negligible. Therefore, in practical applications, rounding is adopted. This can avoid 8-bit integer overflow, that is, avoid Overflow occurred.

[0066] In some feasible implementations, step S9, locating the coordinates of the target pixel in the extended source image based on the integer part, includes: step S901, for each target pixel coordinate within the target image region Rt... Read the corresponding 4 bytes of data from the LUT table, including: , , and Step S902, according to , Locate extended source image The four neighboring pixels include: ; ; ; .

[0067] Specifically, assuming the original source image has integer coordinates (2,3), we determine the four neighboring points of the source image required for bilinear interpolation (all within the expanded image): P00: top-left neighbor, coordinates (2,3), pixel value set to 100; P01: top-right neighbor, coordinates (3,3), pixel value set to 200; P10: bottom-left neighbor, coordinates (2,4), pixel value set to 150; P11: bottom-right neighbor, coordinates (3,4), pixel value set to 250. Because a 1-pixel mirror expansion has been performed, even if the reference coordinates are close to the edge of the original image, the neighboring points will not exceed the range. For example, if the reference coordinates are (0,0), the expanded neighboring points such as (-1,0) will still be valid.

[0068] Based on this, combined Figure 3 , Figure 4 and Figure 5 As shown, compared with the prior art, this embodiment effectively solves the problem of jagged edges and holes in the projected image by performing 1-pixel mirror expansion and inverse transformation.

[0069] In some feasible implementations, step S10, calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm with pure integer operations based on the coordinates of the four neighboring pixels and the fractional part, includes: step S1001, based on the... and The four coefficients of the bilinear interpolation are calculated, including: ; Step S1002: Read the pixel values ​​of the four neighboring pixels, multiply them by W0, W1, W2 and W3 respectively, and sum them to obtain the total weighted value; Step S1003: Normalize the total weighted value by 16 bit shifting to obtain the pixel value of the target pixel after real-time interpolation.

[0070] Specifically, assuming After quantification , After quantification Calculate the four coefficients of bilinear interpolation: ; The result is obtained by multiplying the values ​​of four neighboring pixels by their corresponding coefficients and then summing them up. Because the coefficients were magnified during calculation. =65536 times, the final result needs to be right-shifted by 16 bits (equivalent to dividing by 65536), that is That is, the pixel value of the target pixel after real-time interpolation is 150. The whole process is completed only through integer operations, without floating-point calculations and boundary judgments. And because it is based on an extended image, it avoids holes and jagged edges.

[0071] In some feasible implementations, in step S503, the coordinates of the original source image are... The coordinates are obtained by homogeneous coordinate normalization. , Following this is also: [the following is a list of items / items] and Clamping is performed using the Clamp function to prevent the image from exceeding the boundaries of the extended source image, including: ; ;in, This refers to the dimensions of the original source image. In other words, the calculated coordinates are clamped to ensure they do not exceed the boundaries of the expanded image (although theoretically the expanded image should not exceed the boundaries, numerical calculations may have slight errors).

[0072] In some feasible implementations, to intuitively demonstrate the advantages of this embodiment, a comparative experiment was conducted with the prior art:

[0073] Processing on an S32K344 with a main frequency of 160MHz The images were compared, and the results are shown in Table 1:

[0074] Table 1:

[0075]

[0076] The above implementation determines the minimum bounding rectangle of the trapezoidal projection region through physical constraint modeling, and processes only pixels within this region, reducing computational load. It uses a 1-pixel mirror to expand the original image, ensuring that all neighboring points required for interpolation are within the expanded image, eliminating boundary holes and jagged edges. It pre-calculates and stores the source image coordinates and interpolation weights using a LUT, combined with integer-based real-time interpolation processing, avoiding floating-point operations. This invention can achieve real-time processing of ≥30fps on embedded platforms, effectively improving projected image quality and processing efficiency.

[0077] Example 2

[0078] Please see Figure 6 This embodiment provides a schematic diagram of a projection image geometric correction system based on dynamic physical constraints.

[0079] As an example, the system is implemented using the projection image geometric correction method based on dynamic physical constraints described in Embodiment 1. The system includes a real-time processing system 1 and a pre-computation system 2.

[0080] The pre-computation system 2 integrates a homography parameter spatial discretization unit 20, a LUT generation engine 21, and a LUT compression storage unit 22. The real-time processing system 1 integrates an image input unit 10, a boundary expansion unit 11, a physical constraint processor 12, a LUT indexing engine 13, an integer difference unit 14, and a corrected image output unit 15.

[0081] The image input unit 10 is used to receive the input raw source image.

[0082] The physical constraint processor 12 is used to acquire the pose data of the projection device in real time through the vehicle body sensor, establish a dynamic trapezoidal domain of the projection area based on the pose data, and calculate the minimum bounding rectangle of the dynamic trapezoidal domain as the target image area.

[0083] The boundary expansion unit 11 is used to perform boundary expansion processing on the original source image to obtain an expanded source image.

[0084] The homography parameter spatial discretization unit 20 is used to obtain the inverse perspective transformation matrix based on the spatial discretization of the projection device parameters. .

[0085] The LUT generation engine 21 is used to traverse each pixel in the target image region according to the inverse perspective transformation matrix, calculate its corresponding mapping coordinates in the extended source image, decompose the corresponding mapping coordinates in the extended source image into an integer part representing the coordinates of the upper left reference pixel in the original source image and a fractional part representing the offset from the upper left reference pixel, and quantize the fractional part into Q8 format.

[0086] The LUT compression storage unit 22 is used to pack and compress the integer part and the fractional part into an N-byte data structure, and store it sequentially according to the position of each pixel in the target image area to form a LUT table.

[0087] The LUT index engine 13 is used to call the LUT table to read the LUT entries corresponding to the target pixels in the target image region, obtain the corresponding integer part and fractional part, and locate the coordinates of the four neighboring pixels in the extended source image based on the integer part.

[0088] The integer difference unit 14 is used to calculate the brightness or color intensity value of the target pixel in the corrected image based on the coordinates of the four neighboring pixels and the fractional part using a bilinear interpolation algorithm with pure integer operations.

[0089] The corrected image output unit 15 is used to output the corrected image.

[0090] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0091] It is worth noting that all modules involved in this embodiment are logical units. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0092] Example 3

[0093] Please see Figure 7 The present invention also provides an electronic device, including: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the projection image geometric correction method based on dynamic physical constraints provided in Embodiment 1.

[0094] The memory 702 and processor 701 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 701 and memory 702 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 701 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 701.

[0095] Processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 702 can be used to store data used by processor 701 during operation.

[0096] Example 4

[0097] This invention also proposes a storage medium storing a projection image geometric correction method based on dynamic physical constraints. When the projection image geometric correction program based on dynamic physical constraints is executed by a processor, it implements the steps of the projection image geometric correction method based on dynamic physical constraints as described above. Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0098] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for geometric correction of projected images based on dynamic physical constraints, characterized in that, The method includes: Step S1: Acquire the pose data of the projection device in real time through the vehicle body sensors; Step S2: Establish a dynamic trapezoidal domain for the projection area based on the pose data; Step S3: Calculate the minimum bounding rectangle of the dynamic trapezoidal domain as the target image region; Step S4: Perform boundary expansion processing on the original source image to obtain the expanded source image; Step S5: Obtain the inverse perspective transformation matrix by discretizing the projection device parameters, traverse each pixel in the target image region, and calculate its corresponding mapping coordinates in the extended source image; Step S6: Decompose the corresponding mapped coordinates in the extended source image into integer and fractional parts; Step S7: Quantize the fractional part into Q8 format, pack and compress the integer part and fractional part into an N-byte data structure, and store it in order according to the position of each pixel in the target image region to form a LUT table; Step S8: Call the LUT table to read the LUT entries corresponding to the target pixels in the target image region, and obtain the corresponding integer part and fractional part; Step S9: Locate the coordinates of the four neighboring pixels of the target pixel in the extended source image based on the integer part; Step S10: Calculate the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm with pure integer operations based on the coordinates of the four neighboring pixels and the fractional part. Step S11: Repeat steps S8-S10 by traversing all target pixels within the target image region to output the corrected image.

2. The geometric correction method for projected images based on dynamic physical constraints according to claim 1, characterized in that, Step S3, calculating the minimum bounding rectangle of the dynamic trapezoidal domain as the target image region, includes: Step S301: Using homography transformation, the rectangular original source image projection area is mapped to the dynamic trapezoidal area in the actual scene; Step S302: Calculate the coordinates of the four vertices of the trapezoidal region; Step S303: Obtain the minimum and maximum values ​​of all vertex coordinates in the X-axis and Y-axis directions; Step S304: Obtain the target image region Rt based on the minimum and maximum values: in, .

3. The geometric correction method for projected images based on dynamic physical constraints according to claim 1, characterized in that, Step S4, performing boundary expansion processing on the original source image to obtain an expanded source image, includes: expanding the original source image by one pixel on each side, with the pixel values ​​of the expanded portion filled using a mirror image method to form the expanded source image. .

4. The geometric correction method for projected images based on dynamic physical constraints according to claim 1, characterized in that, Step S5, obtaining the inverse perspective transformation matrix based on the spatial discretization of the projection device parameters, and traversing each pixel in the target image region to calculate its corresponding mapped coordinates in the extended source image, includes: Step S501: Obtain the inverse perspective transformation matrix based on the spatial discretization of the projection device parameters. ; Step S502: Traverse each target pixel within the target image region Rt. According to the inverse perspective transformation matrix Convert it to the original source image coordinates ; Step S503: Assess the coordinates of the original source image. The coordinates are obtained by homogeneous coordinate normalization. , ); Step S504: Set the coordinates of the original source image The coordinate reference system is translated from the original source image to the extended source image to obtain its corresponding mapped coordinates in the extended source image. .

5. The geometric correction method for projected images based on dynamic physical constraints according to claim 4, characterized in that, Step S6, decomposing the corresponding mapping coordinates in the extended source image into integer and fractional parts, includes: Step S601: Use the floor function to... Decompose into integer parts ,in , ; Step S602, to Decompose into decimal parts ,in , .

6. The geometric correction method for projected images based on dynamic physical constraints according to claim 5, characterized in that, Step S7 involves quantizing the fractional part into Q8 format, packaging and compressing the integer and fractional parts into an N-byte data structure, storing it sequentially according to the pixel position in the target image region, forming a LUT table, including: Step S701: Use the Round function to round the decimal part. and Quantized into 8-bit integers respectively and ,in, , ; Step S702, to , , and The data is packaged and compressed into a 32-bit data structure, using a 9-bit x-coordinate and a 7-bit y-coordinate bit field design, combined with an 8-bit Q8 weight, to form a 4-byte or pixel LUT storage structure.

7. The geometric correction method for projected images based on dynamic physical constraints according to claim 6, characterized in that, Step S9, locating the coordinates of the target pixel in the extended source image based on the integer part, includes: Step S901: For the coordinates of each target pixel within the target image region Rt Read the corresponding 4 bytes of data from the LUT table, including: , , and ; Step S902, according to , Locate extended source image The four neighboring pixels include: ； ； ； 。 8. The geometric correction method for projected images based on dynamic physical constraints according to claim 7, characterized in that, Step S10, calculating the brightness or color intensity value of the target pixel in the corrected image using a bilinear interpolation algorithm with pure integer operations based on the coordinates of the four neighboring pixels and the fractional part, includes: Step S1001, based on the above and The four coefficients of the bilinear interpolation are calculated, including: ； ； Step S1002: Read the pixel values ​​of the four neighboring pixels, multiply them by W0, W1, W2 and W3 respectively, and sum them to obtain the total weighted value; Step S1003: Normalize the total weighted value by 16-bit shift to obtain the pixel value of the target pixel after real-time interpolation.

9. The geometric correction method for projected images based on dynamic physical constraints according to claim 4, characterized in that, In step S503, the coordinates of the original source image are... The coordinates are obtained by homogeneous coordinate normalization. , Following this is also: Regarding the and Clamping is performed using the Clamp function to prevent the image from exceeding the boundaries of the extended source image, including: ； ； in, This refers to the dimensions of the original source image.

10. A projection image geometric correction system based on dynamic physical constraints, implemented using the projection image geometric correction method based on dynamic physical constraints as described in any one of claims 1-9, characterized in that, The system includes a real-time processing system and a pre-computation system; The pre-computation system integrates a homography parameter spatial discretization unit, a LUT generation engine, and a LUT compression and storage unit. The real-time processing system integrates an image input unit, a boundary expansion unit, a physical constraint processor, a LUT indexing engine, an integer difference unit, and a corrected image output unit. The image input unit is used to receive the input raw source image; The physical constraint processor is used to acquire the pose data of the projection device in real time through the vehicle body sensor, establish a dynamic trapezoidal domain of the projection area based on the pose data, and calculate the minimum bounding rectangle of the dynamic trapezoidal domain as the target image area. The boundary expansion unit is used to perform boundary expansion processing on the original source image to obtain an expanded source image; The homography parameter spatial discretization unit is used to obtain the inverse perspective transformation matrix based on the spatial discretization of the projection device parameters. ; The LUT generation engine is used to traverse each pixel in the target image region according to the inverse perspective transformation matrix, calculate its corresponding mapping coordinates in the extended source image, decompose the corresponding mapping coordinates in the extended source image into an integer part representing the coordinates of the upper left reference pixel in the original source image and a fractional part representing the offset from the upper left reference pixel, and quantize the fractional part into Q8 format. The LUT compression storage unit is used to pack and compress the integer part and the fractional part into an N-byte data structure, and store it sequentially according to the position of each pixel in the target image region to form a LUT table; The LUT indexing engine is used to call the LUT table to read the LUT entries corresponding to the target pixels in the target image region, obtain the corresponding integer part and fractional part, and locate the coordinates of 4 neighboring pixels in the extended source image based on the integer part; The integer difference unit is used to calculate the brightness or color intensity value of the target pixel in the corrected image based on the coordinates of the four neighboring pixels and the fractional part using a bilinear interpolation algorithm with pure integer operations. The corrected image output unit is used to output the corrected image.

Images