Light carpet projection control method, device, vehicle and program product

Abstract

The application relates to a light blanket projection control method, device, vehicle and program product. The method comprises the following steps: acquiring initial projection parameters corresponding to a current projection frame of a projection lamp, and acquiring a projection parameter prediction drift direction and a projection parameter prediction error increment associated with the current projection frame; acquiring a projection parameter actual error increment corresponding to the current projection frame according to actual projection parameters; determining a projection parameter compensation corresponding to the current projection frame according to the projection parameter prediction drift direction and the difference between the projection parameter prediction error increment and the projection parameter actual error increment; updating the initial projection parameters of the current projection frame; and returning to the step of acquiring the projection parameter prediction drift direction and the projection parameter prediction error increment associated with the current projection frame based on the updated initial projection parameters. The method can adapt to the response requirement of the light blanket following characteristic, and improve the robustness of the light blanket dynamic projection effect.

Description

Technical Field

[0001] This application relates to the field of vehicle lighting control technology, and in particular to a light carpet projection control method, device, vehicle, and computer program product. Background Technology

[0002] DLP (Digital Light Processing Headlights) systems are not only lighting tools for automobiles, but also a "language" for automobiles to communicate with the environment and pedestrians, driving the development of automotive lighting from functionality to intelligence and personalization.

[0003] Light carpet projection is one of the main projection functions of a DLP system. It uses a DLP projector to modulate the light beam and project the image onto the ground, creating a visible light strip or pattern that can serve as a road guide, boundary warning, and ambient lighting.

[0004] In related technologies, the calibration and adjustment of the projection parameters of DLP projectors rely on static preset geometric models. However, this method is difficult to adapt to the motion characteristics of the light carpet (i.e., the light carpet moves with the vehicle), which may cause display abnormalities such as nonlinear distortion of the light carpet, affecting the projection effect of the light carpet and related visual assistance functions. Summary of the Invention

[0005] Therefore, it is necessary to provide a robust light carpet projection control method, device, vehicle, and computer program product that can adapt to the response requirements of the light carpet's motion characteristics and improve the dynamic projection effect of the light carpet, in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a method for controlling the projection of a light carpet, comprising:

[0007] Obtain the initial projection parameters of the projection lamp corresponding to the current projection frame, and obtain the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame;

[0008] The light carpet projection is controlled according to the initial projection parameters to obtain the actual projection parameters of the current projection frame, and the actual error increment of the projection parameters corresponding to the current projection frame is obtained based on the actual projection parameters.

[0009] Based on the predicted drift direction of the projection parameters, and the comparison between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and the preset error increment threshold, the projection parameter compensation amount corresponding to the current projection frame is determined.

[0010] Using the projection parameter compensation amount of the current projection frame, update the initial projection parameters of the current projection frame, use the updated initial projection parameters as the initial projection parameters of the next projection frame, and use the next projection frame as the new current projection frame. Then return to execute the steps of obtaining the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

[0011] This technical solution has the following technical effects:

[0012] By acquiring the predicted drift direction and prediction error increment of the current projection frame in real time, and calculating the actual error increment in combination with the actual projection parameters, the compensation amount of the projection parameters is accurately determined. By updating the initial projection parameters frame by frame and iteratively fine-tuning them, the system dynamically adapts to the changes in the light carpet caused by vehicle movement, effectively suppressing display anomalies such as nonlinear distortion during the light carpet movement process, improving the robustness of the light carpet projection effect, and ensuring that the visual assistance function of the light carpet can be stably performed as much as possible.

[0013] In an optional embodiment of the first aspect, the method further includes:

[0014] In the previous projection frame of the current projection frame, predict the projection parameter prediction drift direction and the projection parameter prediction error increment associated with the current projection frame.

[0015] The prediction methods for projection parameter prediction drift direction and projection parameter prediction error increment include:

[0016] Acquire vehicle motion status data, as well as historical reprojection errors of the projection lights;

[0017] Input the historical reprojection error and vehicle motion state data into the pre-trained projection parameter prediction model;

[0018] By using the projection parameter prediction model, the temporal variation characteristics of historical reprojection errors and the motion state characteristics of vehicle motion state data are extracted, and the temporal variation characteristics and motion state characteristics are fused to obtain the target fusion characteristics.

[0019] The projection parameter prediction model outputs the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame based on the target fusion features.

[0020] This technical solution has the following technical effects:

[0021] By extracting the temporal variation characteristics of historical reprojection errors through a projection parameter prediction model, these temporal variation characteristics can reflect the reprojection drift trend (including magnitude and direction). Combined with the motion state characteristics extracted from real-time vehicle motion state data, a target fusion feature is obtained, which can express multi-dimensional correlation information. In the previous frame of the current frame, the drift direction and error increment of the projection parameters of the current frame can be accurately predicted, providing a reliable data basis for determining the compensation amount of the projection parameters corresponding to the current frame. This fully adapts to the following characteristics of the light carpet and dynamically fine-tunes the projection parameters, so that the light carpet can maintain the best possible visual effect.

[0022] In an optional embodiment of the first aspect, the motion state features include vehicle speed features and vibration features. By fusing temporal variation features and motion state features, target fusion features are obtained, including:

[0023] By fusing vehicle speed characteristics and vibration characteristics, a vehicle speed-vibration fused characteristic is obtained.

[0024] Determine the correlation features between vehicle speed vibration fusion features and temporal variation features; the correlation features are used to characterize the relationship between the reprojection error of the projection lamp, vehicle speed, and vehicle vibration.

[0025] By fusing correlation features and temporal variation features, the target fusion feature is obtained.

[0026] This technical solution has the following technical effects:

[0027] By fusing the correlation features between vehicle speed vibration fusion features and temporal variation features, as well as the temporal variation features, the resulting target fusion features can characterize the multi-dimensional fusion information of vehicle speed, vibration, and error drift. This further adapts to the changes in complex working conditions during the light carpet homing process, effectively improving the accuracy of predicting the drift direction and the error increment of the projection parameters.

[0028] In an optional embodiment of the first aspect, when the current projection frame is the first projection frame, obtaining the initial projection parameters of the projection lamp corresponding to the current projection frame includes:

[0029] Control the projection lamp to project the test pattern, and obtain the reference projection parameters of the projection lamp based on the test pattern;

[0030] The reference projection parameters are input into the pre-built projection geometry model of the projection lamp, and the predicted projection pixel coordinates of the test pattern are output through the projection geometry model.

[0031] Obtain the actual projected pixel coordinates of the test pattern, input the actual projected pixel coordinates and the predicted projected pixel coordinates into the pre-trained error correction model, and output the prediction residual of the projection geometry model through the error correction model.

[0032] The reference projection parameters are optimized based on the predicted residuals to obtain optimized projection parameters. These optimized projection parameters are then used as new reference projection parameters. The process of inputting the reference projection parameters into the pre-built projection geometry model of the projection luminaire is repeated until the optimized projection parameters meet the optimization stopping condition, thus obtaining the initial projection parameters.

[0033] This technical solution has the following technical effects:

[0034] Self-supervised projection parameter calibration is performed by controlling the projection lamp to project the test pattern. It does not rely on manual calibration plates or markers. It uses the projection geometry model and error correction model to perform closed-loop optimization to perform initial calibration of the projection parameters, providing more accurate initial projection parameters for the first projection frame. This provides reliable data support for online fine-tuning of the initial projection parameters for subsequent projection frames.

[0035] In an optional embodiment of the first aspect, outputting the predicted projected pixel coordinates of the test pattern through a projection geometry model includes:

[0036] By using the projection geometry model and according to the preset planar projection constraints, the projected pixel coordinates of the test pattern on the plane are predicted, and the predicted projected pixel coordinates are obtained.

[0037] This technical solution has the following technical effects:

[0038] By adding planar projection constraints to the planar projection characteristics of the light carpet, unlike projecting arbitrary three-dimensional points in space, the solution dimensionality can be significantly reduced and converged faster. It can also avoid spatial noise interference and reduce error jumps introduced by non-ground points caused by vehicle vibration. This makes the projection geometry model lighter and can predict the pixel coordinates of the test pattern more efficiently.

[0039] In an optional embodiment of the first aspect, the method further includes:

[0040] Control the projection lamps to project a verification texture pattern onto the target area; the target area is the area where the projection lamps have already projected the light blanket.

[0041] If the distortion error and / or positional deviation between the verified texture pattern and the preset texture pattern do not meet the corresponding preset conditions, determine the local area in the target area that is associated with the distortion error and / or positional deviation.

[0042] The projection parameters of the projection lights associated with the local area are recalibrated.

[0043] This technical solution has the following technical effects:

[0044] By verifying the overall distortion error and / or positional deviation between the texture pattern and the preset texture pattern in a closed loop during the light carpet projection process, the local areas associated with the distortion error and / or positional deviation are located, and the projection parameters associated with the local areas are dynamically recalibrated locally. This ensures that the projection display effect of the light carpet is not affected by factors under complex working conditions, effectively improving the robustness of the DLP projection system.

[0045] In an optional embodiment of the first aspect, recalibrating the projection parameters of projection lamps associated with a local area includes:

[0046] Control the projection lights to project and verify local texture patterns in a local area;

[0047] With the goal of verifying that the distortion error and / or positional deviation of the local texture pattern and the preset local texture pattern meet the corresponding preset conditions, the projection parameters of the projection lamps associated with the local area are recalibrated.

[0048] In an optional embodiment of the first aspect, the projection parameter compensation amount has a corresponding value and a compensation direction. Based on the predicted drift direction of the projection parameters and a comparison between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and a preset error increment threshold, the projection parameter compensation amount corresponding to the current projection frame is determined, including:

[0049] Based on the predicted drift direction using projection parameters, determine the compensation direction for the projection parameter compensation amount.

[0050] If the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters is less than or equal to the preset error increment threshold, the value of the projection parameter compensation amount is determined based on the predicted error increment of the projection parameters.

[0051] If the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters is greater than the preset error increment threshold, real-time vehicle motion state data is obtained, and the predicted error increment of the projection parameters is corrected based on the real-time vehicle motion state data.

[0052] Based on the predicted error increment of the corrected projection parameters, determine the value of the projection parameter compensation.

[0053] This technical solution has the following technical effects:

[0054] Dedicated calibration textures are projected only onto local areas where distortion errors and / or positional deviations are abnormally correlated, eliminating the need for full-area calibration of the entire light carpet and reducing computational and control overhead. At the same time, local recalibration of projection parameters is performed with the local texture distortion and / or positional deviation meeting the standard as a constraint, accurately correcting local projection distortion and positional offset, avoiding overcorrection problems caused by global parameter adjustment, and effectively improving the consistency and image regularity of local projection of the vehicle-mounted DLP light carpet.

[0055] Secondly, this application also provides a light carpet projection control device, comprising:

[0056] The first acquisition module is used to acquire the initial projection parameters of the projection lamp corresponding to the current projection frame, and to acquire the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

[0057] The second acquisition module is used to control the light carpet projection of the current projection frame according to the initial projection parameters, so as to obtain the actual projection parameters of the current projection frame, and to obtain the actual error increment of the projection parameters corresponding to the current projection frame based on the actual projection parameters.

[0058] The determination module is used to determine the projection parameter compensation amount corresponding to the current projection frame based on the predicted drift direction of the projection parameters and the comparison between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and the preset error increment threshold.

[0059] The iteration module is used to update the initial projection parameters of the current projection frame using the projection parameter compensation amount of the current projection frame, and use the updated initial projection parameters as the initial projection parameters of the next projection frame of the current projection frame, and use the next projection frame of the current projection frame as the new current projection frame, and return to execute the steps of obtaining the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

[0060] Thirdly, this application also provides a vehicle including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0061] Fourthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above aspects.

[0062] Regarding the technical effects of any of the technical solutions in the second to fourth aspects mentioned above, refer to the technical effects of the corresponding technical solutions in the first aspect; repeated examples will not be listed here. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of an optional flow of a light carpet projection control method in one embodiment.

[0065] Figure 2 This is a schematic diagram of the system architecture of the light carpet projection control system in one embodiment.

[0066] Figure 3 This is a schematic diagram of an optional structure of the light carpet projection control device in one embodiment.

[0067] Figure 4 This is a schematic diagram of an optional structure of a vehicle in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0070] In one embodiment, such as Figure 1 As shown, a light carpet projection control method is provided. This embodiment illustrates the application of this method to a terminal (such as the main controller of a vehicle infotainment system, the domain controller of a DLP system, etc.). It is understood that this method can also be applied to a server (e.g., a server remotely controlling a vehicle's light carpet), and can also be applied to a system including both a terminal and a server, implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0071] Step 101: Obtain the initial projection parameters of the projection lamp corresponding to the current projection frame, and obtain the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

[0072] The projection fixtures can be DLP projectors or other lamps mounted on vehicles for light carpet projection. The initial projection parameters of the current projection frame refer to the projection extrinsic and / or projection intrinsic parameters of the projection fixtures used for projection control in the current projection frame. The projection intrinsic parameters may include equivalent projection focal length parameters, projection principal point coordinates, lens distortion coefficients, and physical dimensions and aspect ratios of DLP pixels. The projection extrinsic parameters may include rotation parameters (such as pitch angle, yaw angle, and roll angle) and translation parameters (such as the translation components of the DLP projection optical center in the three-dimensional spatial coordinate system: lateral offset Tx, longitudinal offset Ty, and height offset Tz).

[0073] Among them, the predicted drift direction of the projection parameters refers to the prediction result of the drift direction of the projection parameters of the current projection frame, such as predicting whether the pitch angle drifts in a positive direction (the lens tilts upward) or a negative direction (the lens tilts downward).

[0074] The projection parameter prediction error increment refers to the predicted difference between the projection parameter error of the current projection frame and the projection parameter error of the previous projection frame. For example, for the height offset Tz of the DLP projection optical center, the error in the (n-1)th projection frame is -0.0018m, and the predicted error in the nth projection frame is -0.002m. Therefore, the projection parameter prediction error increment for the nth projection frame is -0.0002m, indicating that the error of the height offset Tz has increased by 0.0002m in the negative direction, where n is a positive integer.

[0075] In practice, the predicted drift direction and prediction error increment of the projection parameters associated with the current projection frame can be predicted in the previous projection frame by combining the vehicle's real-time motion state data and the historical error data of the projection lights.

[0076] In some embodiments, the method further includes:

[0077] In the previous projection frame of the current projection frame, predict the projection parameter prediction drift direction and the projection parameter prediction error increment associated with the current projection frame.

[0078] The prediction methods for projection parameter prediction drift direction and projection parameter prediction error increment include:

[0079] Acquire vehicle motion status data, as well as historical reprojection errors of the projection lights;

[0080] Input the historical reprojection error and vehicle motion state data into the pre-trained projection parameter prediction model;

[0081] By using the projection parameter prediction model, the temporal variation characteristics of historical reprojection errors and the motion state characteristics of vehicle motion state data are extracted, and the temporal variation characteristics and motion state characteristics are fused to obtain the target fusion characteristics.

[0082] The projection parameter prediction model outputs the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame based on the target fusion features.

[0083] The vehicle motion status data can include IMU (Inertial Measurement Unit) vibration data, vehicle speed data, and steering angle data, which can reflect the vehicle's motion status.

[0084] Among them, historical reprojection error refers to the error (residual) between the predicted projection pixel coordinates (e.g., the predicted projection pixel coordinates output based on the projection geometry model (used to reflect the mapping relationship between the pixels of the projection lamp and the light, which can be obtained by mathematical modeling) and the actual projection pixel coordinates (e.g., the actual projection pixel coordinates collected by the ToF (Time-of-Flight) camera) of the light carpet corresponding to the historical projection frame (such as the first N projection frames). The reprojection errors of multiple historical projection frames can constitute a historical reprojection error sequence.

[0085] Among them, the projection parameter prediction model can adopt a 1D (1-dimensional)-CNN (Convolutional Neural Network) network structure.

[0086] In some examples, the projection parameter prediction model can be trained using supervised multi-task regression. Specifically, vehicle motion state data during actual driving and the reprojection error corresponding to the DPL projector on the vehicle are first collected. These data are sampled according to a preset time window to construct training samples including historical reprojection error sequences and vehicle motion state sequences, and the actual projection parameter drift direction and the actual reprojection error change are used as supervision labels. Subsequently, the temporal feature extraction module in the projection parameter prediction model (such as a 1D-CNN network structure) extracts the temporal change features of the historical reprojection error sequence and the motion state features of the vehicle motion state sequence from the training samples. The target fusion features are obtained by feature concatenation or attention mechanism. Then, two independent output heads predict the drift direction and error increment of the projection parameters based on the target fusion features. The end-to-end model parameters are tuned using the joint loss of the two independent output heads (e.g., the classification loss or vector regression loss of the projection parameter drift direction and the regression loss of the projection parameter error increment). The training stops when the training conditions are met (e.g., model convergence or the number of training iterations reaches a preset threshold). This enables the projection parameter prediction model to have a stable and accurate ability to predict the trend of projection parameter changes.

[0087] In the specific implementation, vehicle motion state data and historical reprojection errors are preprocessed and spatiotemporally aligned. A multi-source data synchronization protocol (synchronization accuracy ≤1ms) is used to eliminate temporal discrepancies between multi-source data. The historical reprojection error can be a temporal error sequence formed by extracting the reprojection errors of the first 10-20 projection frames. The preprocessed vehicle motion state data and historical reprojection errors are then input into the projection parameter prediction model. The projection parameter prediction model can have a hierarchical feature extraction structure to achieve deep fusion of multi-source data features.

[0088] In the underlying feature extraction structure of the projection parameter prediction model, based on the temporal error sequence, the increasing / decreasing pattern and fluctuation frequency of the error in the time dimension are captured by a 1D convolution kernel (convolution kernel size 3, stride 1), which corresponds to the cumulative trend of projection parameter drift (for example, the error continues to shift in the positive direction of the X-axis, corresponding to the drift trend of the translation component Tx of the projection extrinsic parameter), thus obtaining the temporal change features.

[0089] In the intermediate layer feature extraction structure of the projection parameter prediction model, motion state features of vehicle motion state data are extracted (e.g., vehicle speed features obtained from vehicle speed data and steering angle features obtained from steering angle data, and these features are converted into vector representations). At the same time, the associated vehicle motion state data can be fused on the feature channel (fusion of vibration features and vehicle speed features).

[0090] In the top-level feature extraction structure of the projection parameter prediction model, the temporal variation features and motion state features are fused. The feature dimension is compressed by pooling operation, the core correlation information is retained, and the fused target fusion feature (which can be represented by a temporal feature vector) is output. This target fusion feature contains the correlation information between the historical trend of error and the vehicle motion state.

[0091] In the core decision layer of the projection parameter prediction model, based on the fused temporal feature vector, two-dimensional prediction is achieved through the fully connected layer of the 1D-CNN network and the activation function (ReLU lightweight activation can be used to reduce computational consumption):

[0092] For projection parameters (such as rotation and translation parameters), the drift direction of each parameter is predicted to obtain the predicted drift direction of the projection parameters. For example, by using the correlation information between the IMU longitudinal angular velocity feature and the residual temporal feature expressed by the target fusion feature, the pitch angle is predicted to drift in the positive direction (lens tilting upward) or the negative direction (lens tilting downward); similarly, by using the correlation information between vehicle speed and steering angle expressed by the target fusion feature, the drift direction of the yaw angle (corresponding to the offset direction of the projector attitude when the vehicle body turns); and similarly, by using the IMU lateral acceleration feature expressed by the target fusion feature, the drift direction of the lateral offset Tx (the lateral offset of the projector caused by the left and right swaying of the vehicle body) is predicted, clarifying the drift trend of each extrinsic parameter and providing directional guidance for subsequent parameter fine-tuning.

[0093] For the nth projection frame, combining information such as the cumulative rate of historical errors and the correlation between vibration intensity and vehicle speed expressed by the target fusion features, the error increment of the projection parameters in the (n+1)th projection frame is predicted, thus obtaining the projection parameter prediction error increment. For example, when the vehicle speed is 60 km / h and the IMU vertical vibration acceleration is 0.8g, the error increment of the pitch angle predicted based on the fusion features is 0.02° / frame, and the error increment of the altitude offset Tz is 0.001m / frame. The error increment prediction accuracy is controlled within ±0.005° (rotation parameter) and ±0.0005m (translation parameter).

[0094] In practical applications, after outputting the prediction results, the projection parameter prediction model can also combine the prediction deviation of the previous frame (i.e., the difference between the error increment of the previous frame prediction and the actual error increment) to fine-tune the current prediction results, avoid the accumulation of errors, and ensure that the prediction results are adapted to the real-time fluctuations of vehicle dynamic conditions (vehicle vibration, vehicle speed changes). At the same time, through the lightweight network structure design (reducing the number of convolutional kernels and simplifying the fully connected layers), the latency of the entire fusion analysis and prediction process is ensured to be ≤10ms, meeting the real-time requirements of subsequent parameter fine-tuning.

[0095] In this embodiment, the temporal variation features of historical reprojection errors are extracted through a projection parameter prediction model, so that the temporal variation features can reflect the reprojection drift trend (including magnitude and direction). Combined with the motion state features extracted from real-time vehicle motion state data, the target fusion features are fused to obtain the target fusion features, which can express multi-dimensional correlation information. Then, in the previous frame of the current frame, the drift direction and error increment of the projection parameters of the current frame are accurately predicted, providing a reliable data basis for determining the projection parameter compensation amount corresponding to the current frame. This fully adapts to the following characteristics of the light carpet and dynamically fine-tunes the projection parameters, so that the light carpet can maintain a good visual effect as much as possible.

[0096] In some embodiments, the motion state features include vehicle speed features and vibration features. By fusing temporal variation features and motion state features, target fusion features are obtained, including:

[0097] By fusing vehicle speed characteristics and vibration characteristics, a vehicle speed-vibration fused characteristic is obtained.

[0098] Determine the correlation features between vehicle speed vibration fusion features and temporal variation features; the correlation features are used to characterize the relationship between the reprojection error of the projection lamp, vehicle speed, and vehicle vibration.

[0099] By fusing correlation features and temporal variation features, the target fusion feature is obtained.

[0100] In practical implementation, vehicle motion state data can include vehicle speed data and vibration data. In the intermediate layer feature extraction structure of the projection parameter prediction model, vehicle speed features are extracted from the vehicle speed data, and vibration features are extracted from the vibration data. The vehicle speed features and vibration features are then concatenated and fused along the feature channels to obtain the vehicle speed-vibration fusion feature. For example, the m-dimensional vehicle speed feature and n-dimensional vibration feature, represented as feature vectors, can be directly concatenated into an m+n-dimensional vehicle speed-vibration fusion feature, where m and n are positive integers. Alternatively, based on an attention mechanism, the weight coefficients of each feature channel of the vehicle speed feature and each feature channel of the vibration feature can be calculated separately, and then the vehicle speed feature and vibration feature can be weighted and fused to obtain the vehicle speed-vibration fusion feature.

[0101] Subsequently, the correlation features between vehicle speed vibration fusion features and temporal variation features are extracted using 1D-CNN convolutional kernels. For example, when the vertical vibration acceleration is detected to be greater than a preset threshold (0.5g) and the vehicle speed is ≥30km / h, the positive / negative correlation coefficients between the projection parameter error and vibration and vehicle speed under this condition are captured. These correlation coefficients are the correlation features between vehicle speed vibration fusion features and temporal variation features, which can quantify the relationship between "vibration intensity-vehicle speed-error drift".

[0102] In the top-level feature extraction structure of the projection parameter prediction model, the correlation features between vehicle speed vibration fusion features and temporal variation features, as well as the temporal variation features of historical reprojection errors, are further fused. The feature dimension is compressed through pooling operations, the core correlation information is retained, and the fused target fusion feature (which can be represented by a temporal feature vector) is output. This target fusion feature contains three types of core information: residual historical trend, vibration interference effect, and vehicle speed condition correlation, so as to characterize the impact of complex conditions on projection parameters.

[0103] In this embodiment, by fusing the correlation features between the vehicle speed vibration fusion features and the temporal variation features, as well as the temporal variation features, the obtained target fusion features can characterize the multi-dimensional fusion information of vehicle speed, vibration, and error drift, further adapting to the changes in complex working conditions during the light carpet follow-up process, and effectively improving the accuracy of the projection parameter prediction drift direction and the projection parameter prediction error increment.

[0104] Step 102: Perform light carpet projection control on the current projection frame according to the initial projection parameters to obtain the actual projection parameters of the current projection frame, and obtain the actual error increment of the projection parameters corresponding to the current projection frame based on the actual projection parameters.

[0105] The actual increment of projection parameter error refers to the actual difference between the projection parameter error of the current projection frame and the projection parameter error of the previous projection frame.

[0106] In practical implementation, the actual projection parameters of the current projection frame can be calculated based on the actual projection onto the ground using a light blanket. For example, ToF point cloud depth data is collected from the projection area of ​​the light blanket, and the actual height offset Tz is calculated based on the ToF point cloud depth data, which is the actual projection parameter. The actual error increment of the projection parameters corresponding to the current projection frame can be calculated by comparing the actual projection parameters of the current projection frame with the actual projection parameters of the previous projection frame. For example, for the height offset Tz of the DLP projection optical center, if the actual error is measured to be -0.0018m based on the actual projection parameters of the (n-1)th projection frame, and the actual error is measured to be -0.00198m based on the actual projection parameters of the nth projection frame, then the actual error increment of the projection parameters of the nth projection frame is -0.00018m, indicating that the error of the height offset Tz has increased by 0.00018m in the negative direction, where n is a positive integer.

[0107] Step 103: Based on the predicted drift direction of the projection parameters and the comparison between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and the preset error increment threshold, determine the projection parameter compensation amount corresponding to the current projection frame.

[0108] Among them, the projection parameter compensation amount is the compensation amount for the initial projection parameters of the current projection frame, which is also called the micro-update amount. It can include the value (the magnitude of the compensation) and the direction (the direction of the compensation, such as rotation or translation).

[0109] In some examples, the predicted drift direction of the projection parameters is a horizontal leftward shift. The difference between the predicted error increment and the actual error increment in the current projection frame is calculated. If the difference is greater than or equal to a preset error increment threshold, the weight of the deviation amplitude is adjusted in combination with the horizontal drift trend. Based on the deviation amplitude and the corresponding weight, the projection parameter compensation amount used to offset the horizontal shift is calculated.

[0110] In some examples, the predicted drift direction of the projection parameters is determined to be vertical offset. The difference between the predicted error increment of the projection parameters in the current frame and the actual error increment is compared. If the difference is greater than or equal to the preset error increment threshold, the corresponding correction rule is matched according to the difference range. Combined with the determined drift direction, the required projection parameter compensation amount for the current projection frame is calculated.

[0111] In practical applications, the projection parameter compensation amount can also be smoothed by using a sliding window (e.g., taking 15 frames of data) to avoid excessive single micro-updates causing light carpet jitter, ensuring stable parameter updates and adapting to the dynamic working conditions of vehicles in motion.

[0112] In some embodiments, the corresponding weights of the projection parameter compensation amount for different types of projection parameters can be assigned in combination with scene features (such as projection distance, ambient light, etc.), and constraints on the adjustment range of the projection lamp hardware can be added (such as pose parameter constraints (to prevent the adjustment range of the projection lamp from being too large), compensation amount amplitude constraints (to prevent the compensation amount from being too large and causing sudden changes), etc.). The projection parameter compensation amount is adaptively determined through each weight and the adjustment range constraints of the projection lamp hardware.

[0113] In some embodiments, the projection parameter compensation amount has a corresponding value and a compensation direction. Based on the predicted drift direction of the projection parameters and a comparison between the difference between the predicted error increment and the actual error increment of the projection parameters and a preset error increment threshold, the projection parameter compensation amount corresponding to the current projection frame is determined, including:

[0114] Based on the predicted drift direction using projection parameters, determine the compensation direction for the projection parameter compensation amount.

[0115] If the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters is less than or equal to the preset error increment threshold, the value of the projection parameter compensation amount is determined based on the predicted error increment of the projection parameters.

[0116] If the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters is greater than the preset error increment threshold, real-time vehicle motion state data is obtained, and the predicted error increment of the projection parameters is corrected based on the real-time vehicle motion state data.

[0117] Based on the predicted error increment of the corrected projection parameters, determine the value of the projection parameter compensation.

[0118] For example, a vehicle's driving condition is as follows: the vehicle is traveling at a constant speed of 40 km / h, and a vertical vibration acceleration of 0.6g is detected in real time (corresponding to slight vibration of the DLP projector).

[0119] Under this driving condition, for the height offset Tz (the height of the DLP projector from the ground) drifting in the negative direction (the direction closer to the ground), the projection parameter prediction error increment is -0.0002m; for the pitch angle of the current projection frame, the projection parameter prediction drift direction is negative (the lens tilts downward), and the projection parameter prediction error increment is -0.003°.

[0120] Based on ToF point cloud depth data, the actual error increment of the projection parameter of height offset Tz is calculated to be -0.00018m. The difference between this and the predicted error increment of the projection parameter is -0.0002m, which is 0.00002m. This is less than the corresponding preset error increment threshold of 0.00005m. Based on the negative direction of the projection parameter prediction drift direction, the compensation direction of the projection parameter compensation is determined to be the positive direction (represented by the sign +). The absolute value of the predicted error increment of the projection parameter, 0.0002m, is taken as the value of the projection parameter compensation of height offset Tz. Therefore, the projection parameter compensation of height offset Tz (the micro-update of the height offset Tz in the current projection frame) is expressed as: +0.0002m.

[0121] The actual error increment of the pitch angle projection parameter is -0.0024°, and the difference between it and the predicted error increment of the projection parameter is -0.003° is 0.0006°, which is greater than the corresponding preset error increment threshold of 0.0005°. Therefore, the vibration intensity data of the vehicle is obtained, and the predicted error increment of the pitch angle projection parameter is corrected to -0.0028°. The corrected predicted error increment of the pitch angle projection parameter is used as the value of the pitch angle projection parameter compensation. At the same time, the compensation direction of the projection parameter compensation is determined to be the positive direction (represented by the symbol +). Therefore, the pitch angle projection parameter compensation is expressed as: +0.0028°.

[0122] For yaw angles, roll angles, lateral offsets Tx, and longitudinal offsets Ty, which have no significant drift (the incremental error of the predicted projection parameter is equal to or the difference between the actual error of the projection parameter is very small), there is no need to calculate the projection parameter compensation.

[0123] In this embodiment, the drift direction is predicted by the projection parameters to clarify the compensation direction and avoid projection anomalies caused by reverse correction. At the same time, the calculation strategy of the compensation value is layered according to the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters. When the difference is small, the calculation is simplified and the response speed is improved. When the difference is large, the error is corrected in combination with the real-time motion status of the vehicle, which effectively suppresses projection drift and accumulated error and ensures the stability of the light carpet projection.

[0124] Step 104: Update the initial projection parameters of the current projection frame using the projection parameter compensation amount of the current projection frame, and use the updated initial projection parameters as the initial projection parameters of the next projection frame of the current projection frame, and use the next projection frame of the current projection frame as the new current projection frame, and return to execute the steps of obtaining the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

[0125] In the specific implementation, the initial projection parameters of the current projection frame can be updated by superimposing the projection parameter compensation amount (including the value and direction of the projection parameter compensation amount), and the updated initial projection parameters can be used for iteration to achieve dynamic updating and correction of the projection parameters.

[0126] In some embodiments, when the current projection frame is the first projection frame, obtaining the initial projection parameters of the projection lamp corresponding to the current projection frame includes:

[0127] Control the projection lamp to project the test pattern, and obtain the reference projection parameters of the projection lamp based on the test pattern;

[0128] The reference projection parameters are input into the pre-built projection geometry model of the projection lamp, and the predicted projection pixel coordinates of the test pattern are output through the projection geometry model.

[0129] Obtain the actual projected pixel coordinates of the test pattern, input the actual projected pixel coordinates and the predicted projected pixel coordinates into the pre-trained error correction model, and output the prediction residual of the projection geometry model through the error correction model.

[0130] The reference projection parameters are optimized based on the predicted residuals to obtain optimized projection parameters. These optimized projection parameters are then used as new reference projection parameters. The process of inputting the reference projection parameters into the pre-built projection geometry model of the projection luminaire is repeated until the optimized projection parameters meet the optimization stopping condition, thus obtaining the initial projection parameters.

[0131] The first projection frame can refer to the projection frame corresponding to the first projection of the light carpet when the light carpet projection function is activated or offline. In such cases, the projection parameters of the projection lamps need to be initially calibrated to determine the initial projection parameters for the first projection frame.

[0132] The test pattern can be a random, textured pattern, and the reference projection parameter refers to the initial value of the projection parameter used for the calibration process when the projection parameter of the projection lamp needs to be calibrated.

[0133] The projection geometry model reflects the mapping relationship between pixels and light rays in the projection lamp. It can be obtained through mathematical modeling. The projection geometry model maps 3D ground points to pixel coordinates, yielding the predicted projected pixel coordinates of the test pattern. The projection geometry model can employ a differentiable and differentiable algorithm structure to receive backpropagation parameters, and a collaborative error correction model performs end-to-end optimization of the reference projection parameters. This projection geometry model can be reused during the subsequent projection of the light carpet. The predicted projected pixel coordinates of the light carpet in the next projection frame are output from the projection geometry model, and then error calculations are performed between these coordinates and the actual projected pixel coordinates obtained from the actual data acquisition to obtain the historical reprojection error.

[0134] The error correction model can adopt a network structure that combines CNN and MLP (Multilayer Perceptron), and can fit the prediction residual of the projection geometry model based on the actual and predicted projection pixel coordinates of the input.

[0135] In the specific implementation, when the current projection frame is the first projection frame, such as when the DLP lighting system is started or offline, the DLP projector is first controlled to project a test pattern with random textures onto the target area. The ToF camera is used to collect the projected image of the test pattern and the scene's 3D point cloud data, and the IMU attitude data and initial position information are collected simultaneously.

[0136] Subsequently, feature point detection is performed on the projected image of the test pattern to generate a feature point set and evaluate the confidence level of each feature point, discarding low-quality feature points with confidence levels below a threshold (e.g., 0.7). Then, based on the feature point disparity (i.e., the positional difference of the same feature point in images from different viewpoints) and 3D point cloud data, the 3D spatial coordinates of each feature point are determined. The preset values ​​of each 3D spatial coordinate and projection parameters are input into the projection geometry model, outputting the theoretical projected luminance of each feature point. A luminance consistency loss function is constructed regarding the projection parameters. This luminance consistency loss function characterizes the deviation between the theoretical and actual projected luminance of each feature point. The solution is iteratively solved with the objective of minimizing the function value of this luminance consistency loss function to obtain the reference projection parameters for the projection lamp.

[0137] The reference projection parameters are input into the projection geometry model, which performs intrinsic parameter transformation. For the normalized camera ray direction, pixel coordinate mapping is performed using projection intrinsic parameters (such as focal length, principal point, and distortion) from the reference projection parameters. Specifically, differentiable forward calculations are performed for the distortion of wide-angle lenses, adaptable to vehicle-mounted, small-volume wide-angle DLP light carpet projection. Furthermore, unlike the nonlinear iteration of distortion optimization in related technologies, the distortion layer calculation is differentiable, allowing for end-to-end compensation in conjunction with the neural network of the error correction model, solving the edge distortion problem of the wide-angle light carpet. Subsequently, the projection geometry model performs extrinsic parameter transformation, converting the ground 3D points in the world coordinate system to the camera coordinate system of the DLP system using projection extrinsic parameters (such as rotation and translation) from the input reference projection parameters. Using the vehicle coordinate system (i.e., the camera coordinate system of the DLP system) as an intermediate reference, real-time data such as vehicle speed and IMU attitude can be directly coupled, facilitating subsequent temporal drift prediction. Finally, using the camera ray direction mapped to pixel coordinates and the ground 3D points after coordinate transformation, the predicted projected pixel coordinates of the test pattern are output.

[0138] Furthermore, the projected image of the test pattern is acquired, and the actual pixel coordinates are calculated. The actual and predicted projected pixel coordinates are then input into an error correction model. The prediction residuals of the projection geometry model are fitted and learned through a CNN+MLP network of the error correction model. Subsequently, based on the prediction residuals, a differentiable total loss function for the projection parameters can be constructed. This total loss function is then used to optimize the reference projection parameters to obtain the optimized projection parameters.

[0139] Backpropagation is performed using optimized projection parameters. End-to-end joint optimization is carried out based on the projection geometry model and error correction model until the optimized projection parameters meet the optimization stopping conditions (such as meeting the optimization convergence conditions, reaching the maximum number of optimization iterations, etc.). The initial calibration results of the projection parameters are obtained, which can be used as the initial projection parameters when the current projection frame is the first projection frame.

[0140] In practical applications, the projection geometry model can also calculate the reprojection error of the test pattern. By inputting the predicted projection pixel coordinates of the test pattern, the real-time ToF camera point cloud depth data, and the texture of the test pattern, the model outputs the photometric consistency loss based on the differentiable photometric consistency loss function. This enables sub-pixel-level differentiable sampling and improves the single-pixel accuracy of the light blanket.

[0141] In this embodiment, the projection parameters are calibrated by controlling the projection lamp to project the test pattern. This eliminates the need for manual calibration plates or markers. The projection geometry model and error correction model are used in synergy for closed-loop optimization to perform initial calibration of the projection parameters. This provides more accurate initial projection parameters for the first projection frame and provides reliable data support for online fine-tuning of the initial projection parameters for subsequent projection frames.

[0142] In some embodiments, outputting the predicted projected pixel coordinates of the test pattern through a projection geometry model includes:

[0143] By using the projection geometry model and according to the preset planar projection constraints, the projected pixel coordinates of the test pattern on the plane are predicted, and the predicted projected pixel coordinates are obtained.

[0144] Among them, the planar projection constraint is used to force the projection to be a ground plane (a plane or an approximate plane with a Z-axis coordinate of 0).

[0145] In the specific implementation, reference projection parameters are input into the projection geometry model. The projection geometry model performs intrinsic parameter transformation, mapping the normalized camera ray direction to pixel coordinates using projection intrinsic parameters (such as focal length, principal point, distortion, etc.) in the reference projection parameters. Specifically, differentiable forward calculations are performed for the distortion of wide-angle lenses. Subsequently, the projection geometry model performs extrinsic parameter transformation, converting the ground 3D points in the world coordinate system to the camera coordinate system of the DLP system using projection extrinsic parameters (such as rotation, translation, etc.) in the input reference projection parameters. Using the vehicle coordinate system (i.e., the camera coordinate system of the DLP system) as an intermediate reference, real-time data such as vehicle speed and IMU attitude can be directly coupled, facilitating subsequent temporal drift prediction. Then, using the camera ray direction mapped to pixel coordinates and the ground 3D points after coordinate transformation, the ground 3D points are projected onto a ground plane based on planar projection constraints. Since the projection surface is the ground plane, the model no longer processes arbitrary points in space, thus enabling faster calculation and output of the predicted projected pixel coordinates of the test pattern.

[0146] In this embodiment, by adding planar projection constraints to the planar projection characteristics of the light carpet, which is different from projecting arbitrary three-dimensional points in space, the solution dimensionality can be greatly reduced and converged faster. It can also avoid spatial noise interference and reduce error jumps introduced by non-ground points caused by vehicle vibration. This makes the projection geometry model lighter and can predict the pixel coordinates of the test pattern more efficiently.

[0147] In some embodiments, the method further includes:

[0148] Control the projection lamps to project a verification texture pattern onto the target area; the target area is the area where the projection lamps have already projected the light blanket.

[0149] If the distortion error and / or positional deviation between the verified texture pattern and the preset texture pattern do not meet the corresponding preset conditions, determine the local area in the target area that is associated with the distortion error and / or positional deviation.

[0150] The projection parameters of the projection lights associated with the local area are recalibrated.

[0151] Among them, the verification texture pattern refers to the textured verification pattern that is actually projected onto the ground, such as a checkerboard texture actually projected onto the ground. The preset texture pattern is a pre-stored texture pattern, such as a preset standard checkerboard texture.

[0152] In practice, while the light carpet has been projected onto the target area, a verification texture pattern that does not affect the normal display of the light carpet can be projected at preset intervals to determine whether the recalibration process is triggered and to perform closed-loop verification on the actual projected light carpet pattern.

[0153] If the overall distortion error and / or positional deviation between the verified texture pattern and the preset texture pattern does not meet the corresponding preset conditions, for example, if the distortion error and / or positional deviation is greater than the corresponding threshold (e.g., 0.01m), then the associated projection parameters need to be recalibrated.

[0154] For example, the target area is divided into multiple local areas, and the local patterns of each local area are compared with the corresponding local patterns of the preset texture pattern. The deviations associated with distortion error and / or position deviation are calculated, thereby determining the local areas associated with distortion error and / or position deviation in the target area.

[0155] Furthermore, by combining the specific locations where distortion errors and / or positional deviations occur locally, and according to pre-established correlations, projection parameters that may cause corresponding anomalies can be matched. For example, distortion errors and / or positional deviations occurring at the right edge of the light carpet may be caused by errors in the yaw angle of the projection extrinsic parameters or the distortion coefficients of the projection intrinsic parameters, in which case these projection parameters need to be recalibrated.

[0156] Furthermore, the projection parameters associated with the local region are recalibrated.

[0157] For example, relevant data for a local region (such as ToF point cloud data, measured coordinates of the test pattern in that local region, etc.) can be collected, a loss function can be constructed, and the constructed projection geometry model and pre-trained error correction model can be called. Other projection parameters besides those associated with the local region can be fixed, and the projection parameters associated with the local region can be set as optimizable projection parameters. End-to-end optimization can be performed using the projection geometry model and the error correction model to recalibrate the projection parameters associated with the local region. Since only the parameter terms corresponding to the deviation are optimized, full recalibration is not required, and the recalibration time is relatively short (≤1s).

[0158] In practical applications, the projection area of ​​the verification texture pattern can be a small percentage (e.g., only 5%-10% of the light carpet projection area), and it should be preferentially projected onto the non-core areas at the edges of the light carpet, without obscuring core content such as the light carpet logo and navigation guides. Simultaneously, it can be configured for staggered projection, meaning each projection is extremely short (e.g., only 10-20ms, far shorter than the human eye's visual persistence time), alternating with the normal display frames of the light carpet, making it difficult for the human eye to detect. It can also be configured for adaptive brightness adjustment, where the verification texture pattern blends seamlessly with the light carpet background, allowing only the ToF camera to accurately capture the image without affecting the user's visual perception. Through DLP single-pixel level control, it projects only onto preset verification pixels, without disrupting the original display integrity of the light carpet.

[0159] In this embodiment, by performing closed-loop verification of the overall distortion error and / or positional deviation between the texture pattern and the preset texture pattern during the light carpet projection process, the local area associated with the distortion error and / or positional deviation is located, and the projection parameters associated with the local area are dynamically recalibrated locally. This ensures that the projection display effect of the light carpet is not affected by factors under complex working conditions, effectively improving the robustness of the DLP projection system.

[0160] In some embodiments, the projection parameters of projection lamps associated with a local area are recalibrated, including:

[0161] Control the projection lights to project and verify local texture patterns in a local area;

[0162] With the goal of verifying that the distortion error and / or positional deviation of the local texture pattern and the preset local texture pattern meet the corresponding preset conditions, the projection parameters of the projection lamps associated with the local area are recalibrated.

[0163] Among them, the local texture pattern verification refers to the local pattern with texture that is actually projected onto a local area associated with distortion error and / or position deviation, such as the local checkerboard texture of the actual projection; the preset local texture pattern is the standard local texture pattern, such as the standard local checkerboard texture.

[0164] In some examples, comparing the collected verification texture pattern with the preset texture pattern, the overall positional deviation between the verification texture pattern and the preset texture pattern is calculated to be 0.015mm, exceeding the preset threshold of 0.01mm, which means it does not meet the corresponding preset conditions. In this case, the deviation is first broken down to locate the source of the local deviation: First, based on the coordinates of each corner point of the checkerboard collected by the ToF camera, a point-by-point comparison is made with the standard corner point coordinates. It is found that the overall positional deviation mainly comes from the local area of ​​the right edge of the light carpet (e.g., deviation of 0.012-0.015mm), the positional deviation of the left edge is smaller (e.g., 0.003-0.005mm, which meets the threshold requirement), and there is no deviation in the middle area; further, combined with the historical parameter drift data of the DLP projector, the positional deviation is located and correlated with the yaw angle of the projection extrinsic parameter (currently offset from the standard value by 0.008°) and the radial distortion coefficient of the projection intrinsic parameter (currently -0.123, standard value -0.12, which is within the normal range of distortion coefficient), and the other parameters are normal. The aforementioned projection geometry model was invoked, with all other intrinsic and extrinsic parameters fixed, except for the yaw angle and radial distortion coefficient, which were set as optimizable variables. ToF point cloud data and measured coordinates of the checkerboard pattern were collected from the right edge of the light carpet, and a local photometric consistency loss function was constructed. The parameters were then fine-tuned through backpropagation using the aforementioned projection geometry model and error correction model. Using the fine-tuned yaw angle, radial distortion coefficient, and other fixed projection parameters, a verification local texture pattern was projected onto a local area on the right edge of the light carpet. The positional deviation between the verification local texture pattern and the preset local texture pattern was verified to be reduced to 0.009mm (≤0.01mm), thus meeting the corresponding preset conditions. This completed the local recalibration of the yaw angle and radial distortion coefficient. The entire recalibration process took approximately 0.8s, meeting the design requirement of ≤1s, and did not affect the normal dynamic display of the light carpet.

[0165] In some examples, the overall distortion error between the verification texture pattern and the preset texture pattern is 0.013mm (exceeding the 0.01mm threshold). The overall distortion error is broken down, and the distortion error corresponding to the area below the light carpet (such as the side near the vehicle body) is measured to be 0.011-0.013mm, while the distortion error corresponding to the area above is ≤0.006mm (meeting the standard). Combined with the real-time vertical vibration data of the IMU, the pitch angle (offset from the standard value of 0.007°) and height offset Tz (offset of -0.0003m) corresponding to the distortion error in the area below the light carpet are located. The local recalibration of projection parameters associated with the area below the light carpet is triggered: all other parameters are fixed, and only the pitch angle and height offset Tz are used as optimization variables. ToF data and checkerboard coordinates of the area below the light carpet are collected, and fine-tuning is performed using the aforementioned projection geometry model and error correction model (pitch angle adjustment of 0.0065°, Tz adjustment of 0.00028m). Based on the fine-tuned projection parameters, a verification local texture pattern is projected onto the area below the light carpet. The distortion error between the verification local texture pattern and the preset local texture pattern is verified to be reduced to 0.009mm (meeting the standard), thus completing the local recalibration of the pitch angle and height offset Tz. The entire local recalibration takes approximately 0.7s, meeting the design requirement of ≤1s, and does not affect the normal dynamic display of the light carpet.

[0166] In this embodiment, a dedicated verification texture is projected only for local areas where distortion error and / or position deviation are abnormally correlated, eliminating the need for full-area calibration of the entire light carpet and reducing computational and control overhead. At the same time, the projection parameters are locally recalibrated based on the constraint that the distortion and / or position deviation of the local texture meets the standard, accurately correcting local projection distortion and position offset, avoiding overcorrection problems caused by full-area parameter adjustment, and effectively improving the consistency and image regularity of local projection of the vehicle-mounted DLP light carpet.

[0167] In some embodiments, a light carpet projection control system is provided, the architecture of which is as follows: Figure 2 As shown, it includes: an AI (Artificial Intelligence) calibration module, a perception fusion module, a DLP control module, a closed-loop verification module, and a storage module.

[0168] The AI ​​calibration module includes a lightweight feature extraction unit, a self-supervised geometric constraint unit, and a hybrid optimization unit.

[0169] The lightweight feature extraction unit uses the MobileNet network combined with an onboard lighting adaptive algorithm to extract lighting robust feature points from the ToF point cloud and the DLP projected random texture pattern.

[0170] Among them, the self-supervised geometric constraint unit generates a supervision signal based on photometric consistency and disparity estimation, without the need for manual annotation.

[0171] The hybrid optimization unit integrates a differentiable projection geometric model and a neural error correction network. The geometric model constructs the projection relationship between the DLP projector and the ground scene, while the neural network specifically learns the nonlinear error compensation amount under vehicle conditions, adapting to parameter fluctuations caused by vehicle vibration and temperature changes. Specifically, the hybrid optimization unit can adopt a joint architecture of a differentiable projection geometric model and a neural error correction model (CNN+MLP composite structure). The differentiable projection geometric model provides physical constraints for vehicle-mounted DLP projection, while the neural error correction model specifically learns the compensation amount for nonlinear distortion, temperature drift, and vibration interference under vehicle scenes, forming a two-way feedback optimization closed loop. The MobileNet feature extraction unit can automatically evaluate feature point ambiguity, occlusion, and illumination adaptability, eliminating feature points with confidence scores below a preset threshold (0.7) to improve calibration robustness.

[0172] The perception fusion module consists of a ToF camera, an inertial measurement unit (IMU), and a body control module (BCM). The ToF camera collects 3D point cloud data of the scene, the IMU detects the device's attitude and vibration information, and the BCM provides vehicle speed, steering angle, and other vehicle dynamic signals. All data are transmitted to the AI ​​calibration module after time synchronization, with a synchronization accuracy of ≤1ms.

[0173] The DLP control module receives calibration parameters (internal parameters, external parameters, and error compensation) output by the AI ​​calibration module, controls the DLP projector to adjust pixel grayscale and projection posture, realizes dynamic follow-up projection of the light carpet, and supports single-pixel level precision control.

[0174] Closed-loop verification module: A preset test pattern is projected through a DLP projector, and the projection result is collected by a ToF camera. The AI ​​module analyzes the degree of pattern distortion and positional deviation. If the deviation exceeds the preset threshold (0.01mm), the automatic recalibration process is triggered to ensure that the calibration accuracy continues to meet the standard.

[0175] Storage module: Stores historical calibration parameters, drift trend data, and AI model parameters. It supports rapid initialization after power failure recovery without the need for recalibration.

[0176] For example, the workflow of a light carpet projection control system may include three stages: self-supervised initialization calibration, online fine-tuning of dynamic parameters, and closed-loop accuracy verification.

[0177] During the self-supervised initialization calibration phase (executed when the DLP system is offline or during its first startup, calibrating the initial projection parameters of the first projection frame), the DLP projector projects a random texture test pattern onto the target area, the ToF camera acquires the projected image and the scene's 3D point cloud, and the perception fusion module simultaneously acquires IMU attitude data and initial position information, which are then input into the AI ​​calibration module.

[0178] In the AI ​​calibration module's workflow, the input layer receives IMU pose data, ToF point cloud data, and other data. The feature extraction unit detects feature points in the projected image, generates a feature point set, evaluates the confidence of each point, and removes low-quality points with a confidence of <0.7. The self-supervised geometric constraint unit constructs a photometric consistency loss function based on feature point disparity and 3D point cloud data, and initially solves for the intrinsic and extrinsic parameters of the DLP. Subsequently, the hybrid optimization unit inputs the initial parameters into the differentiable projective geometric model, and the neural error correction model (using a CNN+MLP structure) calculates the residual between the prediction result and the actual acquired data. It optimizes the projection parameters corresponding to the differentiable projective geometric model through backpropagation, outputs the optimized projection parameters at the output layer, and performs bidirectional feedback optimization loop iterations until the optimization stopping condition is met, obtaining the initial projection parameters of the first projection frame. The initial calibration is completed in ≤5 seconds.

[0179] During the online fine-tuning phase of dynamic parameters (n projection frames after the first projection frame), the perception fusion module collects scene data in real time (frame rate 50-200Hz), including ToF point clouds, IMU vibration data, vehicle speed and steering angle signals; the 1D-CNN temporal network performs fusion analysis on the historical calibration residual sequence, synchronously collected IMU vibration data and vehicle speed signals to predict the external parameter drift trend (rotation / translation direction) and error increment. The network structure is designed to be lightweight to adapt to the computing power of vehicle edge devices; the hybrid optimization unit calculates the parameter micro-update based on the prediction results and real-time perception data, and smoothly updates the calibration parameters through a sliding window (window size 10-20 frames) to avoid parameter abrupt changes that cause light carpet jitter; the DLP control module adjusts the projection posture according to the updated parameters to achieve dynamic tracking of the light carpet, with a single frame parameter update latency of ≤20ms.

[0180] During the closed-loop accuracy verification phase (executed during the projection of the light carpet), the DLP projector intermittently projects a standard calibration pattern (such as a checkerboard texture, which does not affect the normal display of the light carpet). The closed-loop verification module compares the collected calibration pattern with the standard pattern and calculates the distortion error and position deviation. If the deviation is ≤0.01mm, the current calibration parameters are maintained. If the deviation is >0.01mm, a local recalibration process is triggered, which only optimizes the parameter items corresponding to the deviation, without full recalibration. The recalibration time is ≤1s.

[0181] The above workflow, when applied to scenarios involving vehicle projection light carpets, offers the following advantages:

[0182] The scene adaptability is significantly improved. No manual calibration board is required. It supports self-calibration in scenarios without markers and can be adapted to complex dynamic scenarios such as vehicle greeting and outdoor navigation, completely eliminating the dependence on manual operation and reducing on-site deployment costs.

[0183] The dynamic accuracy remains stable. Through AI time-series prediction and online fine-tuning, it corrects parameter deviations caused by equipment drift, changes in illumination, and vibration in real time. The calibration accuracy error is controlled within ±0.01mm, and the light carpet tracking offset is ≤1 pixel, achieving "one-time initialization and long-term self-adaptation".

[0184] Balancing real-time performance and accuracy, the lightweight AI model achieves an inference latency of ≤20ms, meeting the millisecond-level follow-up response requirements of DLP light blankets, while retaining distortion correction functionality, thus resolving the contradiction between accuracy and real-time performance in traditional technologies.

[0185] It exhibits strong robustness under complex working conditions. The AI ​​feature extraction module can effectively cope with interference such as strong light, reflection, and occlusion, improving the feature point detection accuracy to over 95%. Compared with related algorithms, the calibration success rate is improved by over 60% under extreme lighting conditions.

[0186] It boasts excellent hardware compatibility, adapting to edge devices such as ARM (Advanced RISC Machine) and NPU (Neural Processing Unit), requiring no cloud computing support. It can be directly integrated into edge terminals such as vehicle central control and smart projection devices, reducing power consumption by more than 30% and facilitating engineering implementation.

[0187] The embodiments of this application have the following beneficial effects: by acquiring the predicted drift direction and prediction error increment of the current projection frame in real time, and calculating the actual error increment in combination with the actual projection parameters, the compensation amount of the projection parameters is accurately determined; and by updating the initial projection parameters frame by frame and iteratively fine-tuning them, the changes in the light carpet caused by vehicle movement are dynamically adapted, effectively suppressing display anomalies such as nonlinear distortion during the light carpet movement process, improving the robustness of the light carpet projection effect, and ensuring that the visual auxiliary function of the light carpet can be stably performed as much as possible.

[0188] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0189] Based on the same inventive concept, this application also provides a light carpet projection control device for implementing the light carpet projection control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more light carpet projection control device embodiments provided below can be found in the limitations of the light carpet projection control method described above, and will not be repeated here.

[0190] In one exemplary embodiment, such as Figure 3 As shown, a light carpet projection control device 30 is provided, comprising:

[0191] The first acquisition module 301 is used to acquire the initial projection parameters of the projection lamp corresponding to the current projection frame, and to acquire the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

[0192] The second acquisition module 302 is used to perform light carpet projection control on the current projection frame according to the initial projection parameters to obtain the actual projection parameters of the current projection frame, and to obtain the actual error increment of the projection parameters corresponding to the current projection frame based on the actual projection parameters.

[0193] The determination module 303 is used to determine the projection parameter compensation amount corresponding to the current projection frame based on the predicted drift direction of the projection parameters and the comparison result between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and the preset error increment threshold.

[0194] The iteration module 304 is used to update the initial projection parameters of the current projection frame using the projection parameter compensation amount of the current projection frame, and use the updated initial projection parameters as the initial projection parameters of the next projection frame of the current projection frame, and use the next projection frame of the current projection frame as the new current projection frame, and return to execute the steps of obtaining the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

[0195] In some embodiments, the device is also used for:

[0196] In the previous projection frame of the current projection frame, predict the projection parameter prediction drift direction and the projection parameter prediction error increment associated with the current projection frame.

[0197] The prediction methods for projection parameter prediction drift direction and projection parameter prediction error increment include:

[0198] Acquire vehicle motion status data, as well as historical reprojection errors of the projection lights;

[0199] Input the historical reprojection error and vehicle motion state data into the pre-trained projection parameter prediction model;

[0200] By using the projection parameter prediction model, the temporal variation characteristics of historical reprojection errors and the motion state characteristics of vehicle motion state data are extracted, and the temporal variation characteristics and motion state characteristics are fused to obtain the target fusion characteristics.

[0201] The projection parameter prediction model outputs the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame based on the target fusion features.

[0202] In some embodiments, the motion state features include vehicle speed features and vibration features. By fusing temporal variation features and motion state features, target fusion features are obtained, including:

[0203] By fusing vehicle speed characteristics and vibration characteristics, a vehicle speed-vibration fused characteristic is obtained.

[0204] Determine the correlation features between vehicle speed vibration fusion features and temporal variation features; the correlation features are used to characterize the relationship between the reprojection error of the projection lamp, vehicle speed, and vehicle vibration.

[0205] By fusing correlation features and temporal variation features, the target fusion feature is obtained.

[0206] In some embodiments, when the current projection frame is the first projection frame, obtaining the initial projection parameters of the projection lamp corresponding to the current projection frame includes:

[0207] Control the projection lamp to project the test pattern, and obtain the reference projection parameters of the projection lamp based on the test pattern;

[0208] The reference projection parameters are input into the pre-built projection geometry model of the projection lamp, and the predicted projection pixel coordinates of the test pattern are output through the projection geometry model.

[0209] Obtain the actual projected pixel coordinates of the test pattern, input the actual projected pixel coordinates and the predicted projected pixel coordinates into the pre-trained error correction model, and output the prediction residual of the projection geometry model through the error correction model.

[0210] The reference projection parameters are optimized based on the predicted residuals to obtain optimized projection parameters. These optimized projection parameters are then used as new reference projection parameters. The process of inputting the reference projection parameters into the pre-built projection geometry model of the projection luminaire is repeated until the optimized projection parameters meet the optimization stopping condition, thus obtaining the initial projection parameters.

[0211] In some embodiments, the predicted projected pixel coordinates of the test pattern are output through a projection geometry model, including:

[0212] By using the projection geometry model and according to the preset planar projection constraints, the projected pixel coordinates of the test pattern on the plane are predicted, and the predicted projected pixel coordinates are obtained.

[0213] In some embodiments, the device is also used for:

[0214] Control the projection lamps to project a verification texture pattern onto the target area; the target area is the area where the projection lamps have already projected the light blanket.

[0215] If the distortion error and / or positional deviation between the verified texture pattern and the preset texture pattern do not meet the corresponding preset conditions, determine the local area in the target area that is associated with the distortion error and / or positional deviation.

[0216] The projection parameters of the projection lights associated with the local area are recalibrated.

[0217] In some embodiments, the projection parameters of projection lamps associated with a local area are recalibrated, including:

[0218] Control the projection lights to project and verify local texture patterns in a local area;

[0219] With the goal of verifying that the distortion error and / or positional deviation of the local texture pattern and the preset local texture pattern meet the corresponding preset conditions, the projection parameters of the projection lamps associated with the local area are recalibrated.

[0220] In some embodiments, the projection parameter compensation amount has a corresponding value and a compensation direction. Based on the predicted drift direction of the projection parameters and a comparison between the difference between the predicted error increment and the actual error increment of the projection parameters and a preset error increment threshold, the projection parameter compensation amount corresponding to the current projection frame is determined, including:

[0221] Based on the predicted drift direction using projection parameters, determine the compensation direction for the projection parameter compensation amount.

[0222] If the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters is less than or equal to the preset error increment threshold, the value of the projection parameter compensation amount is determined based on the predicted error increment of the projection parameters.

[0223] If the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters is greater than the preset error increment threshold, real-time vehicle motion state data is obtained, and the predicted error increment of the projection parameters is corrected based on the real-time vehicle motion state data.

[0224] Based on the predicted error increment of the corrected projection parameters, determine the value of the projection parameter compensation.

[0225] Each module in the aforementioned light carpet projection control device 30 can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0226] In one exemplary embodiment, a vehicle is provided that can serve as a terminal, and its internal structure diagram can be as follows: Figure 4 As shown, the vehicle includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The vehicle's processor provides computing and control capabilities. The vehicle's memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The vehicle's input / output interface is used for exchanging information between the processor and external devices. The vehicle's communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a light carpet projection control method. The vehicle's display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device.

[0227] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0228] In one exemplary embodiment, a vehicle is provided, including a memory and a processor, the memory storing a computer program that, when executed by the processor, implements the light carpet projection control method described above.

[0229] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the light carpet projection control method described above.

[0230] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, provides the above-described method for controlling the projection of a light carpet.

[0231] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0232] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0233] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0234] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for controlling the projection of a light carpet, characterized in that, The method includes: Obtain the initial projection parameters of the projection lamp corresponding to the current projection frame, and obtain the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame; The current projection frame is controlled by light carpet projection according to the initial projection parameters to obtain the actual projection parameters of the current projection frame, and the actual error increment of the projection parameters corresponding to the current projection frame is obtained according to the actual projection parameters. Based on the predicted drift direction of the projection parameters, and the comparison between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and the preset error increment threshold, the projection parameter compensation amount corresponding to the current projection frame is determined. Using the projection parameter compensation amount of the current projection frame, the initial projection parameters of the current projection frame are updated, and the updated initial projection parameters are used as the initial projection parameters of the next projection frame of the current projection frame. The next projection frame of the current projection frame is used as the new current projection frame. Then, the process returns to the steps of obtaining the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

2. The method according to claim 1, characterized in that, The method further includes: In the previous projection frame of the current projection frame, predict the projection parameter prediction drift direction and the projection parameter prediction error increment associated with the current projection frame; The prediction methods for the projection parameters to predict the drift direction and the projection parameter prediction error increment include: Acquire vehicle motion state data, and the historical reprojection error of the projection lamp; The historical reprojection error and the vehicle motion state data are input into the pre-trained projection parameter prediction model. The temporal variation features of the historical reprojection error and the motion state features of the vehicle motion state data are extracted through the projection parameter prediction model, and the temporal variation features and the motion state features are fused to obtain the target fusion features. Based on the target fusion features, the projection parameter prediction model outputs the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

3. The method according to claim 2, characterized in that, The motion state features include the vehicle speed features and vibration features. The fusion of the temporal variation features and the motion state features to obtain the target fusion features includes: By fusing the vehicle speed feature and the vibration feature, a vehicle speed-vibration fused feature is obtained; Determine the correlation features between the vehicle speed vibration fusion features and the temporal variation features; the correlation features are used to characterize the relationship between the reprojection error of the projection lamp, the vehicle speed, and the vehicle vibration; The target fusion feature is obtained by fusing the correlation feature and the temporal change feature.

4. The method according to any one of claims 1 to 3, characterized in that, When the current projection frame is the first projection frame, obtaining the initial projection parameters of the projection lamp corresponding to the current projection frame includes: Control the projection lamp to project a test pattern, and obtain reference projection parameters of the projection lamp based on the test pattern; The reference projection parameters are input into the pre-constructed projection geometry model of the projection lamp, and the predicted projection pixel coordinates of the test pattern are output through the projection geometry model. Obtain the actual projected pixel coordinates of the test pattern, input the actual projected pixel coordinates and the predicted projected pixel coordinates into a pre-trained error correction model, and output the prediction residual of the projection geometry model through the error correction model; The reference projection parameters are optimized based on the predicted residuals to obtain optimized projection parameters. The optimized projection parameters are then used as new reference projection parameters. The process of inputting the reference projection parameters into the pre-built projection geometry model of the projection lamp is repeated until the optimized projection parameters meet the optimization stopping condition to obtain the initial projection parameters.

5. The method according to claim 4, characterized in that, The step of outputting the predicted projected pixel coordinates of the test pattern through the projected geometry model includes: Using the projection geometry model and according to preset planar projection constraints, the projected pixel coordinates of the test pattern on the plane are predicted, and the predicted projected pixel coordinates are obtained.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The projection lamp is controlled to project a verification texture pattern onto a target area; the target area is the area where the projection lamp has already projected a light blanket. If the distortion error and / or positional deviation between the verified texture pattern and the preset texture pattern do not meet the corresponding preset conditions, determine the local area associated with the distortion error and / or the positional deviation in the target area; The projection parameters of the projection lamps associated with the local area are recalibrated.

7. The method according to claim 6, characterized in that, The recalibration of the projection parameters of the projection lamps associated with the local area includes: Control the projection lamp to project a local texture pattern for verification onto the local area; With the goal of ensuring that the distortion error and / or positional deviation of the verified local texture pattern and the preset local texture pattern meet the corresponding preset conditions, the projection parameters of the projection lamp associated with the local area are recalibrated.

8. The method according to any one of claims 1 to 3, characterized in that, The projection parameter compensation amount has a corresponding value and compensation direction. The determination of the projection parameter compensation amount corresponding to the current projection frame based on the predicted drift direction of the projection parameters and the comparison result between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and a preset error increment threshold includes: Based on the predicted drift direction using the projection parameters, the compensation direction of the projection parameter compensation amount is determined. If the difference between the predicted error increment of the projection parameter and the actual error increment of the projection parameter is less than or equal to a preset error increment threshold, the value of the projection parameter compensation amount is determined based on the predicted error increment of the projection parameter. If the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters is greater than the preset error increment threshold, the real-time vehicle motion state data of the vehicle is obtained, and the predicted error increment of the projection parameters is corrected based on the real-time vehicle motion state data. The value of the projection parameter compensation amount is determined based on the predicted error increment according to the corrected projection parameters.

9. A light carpet projection control device, characterized in that, The device includes: The first acquisition module is used to acquire the initial projection parameters of the projection lamp corresponding to the current projection frame, and to acquire the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame. The second acquisition module is used to perform light carpet projection control on the current projection frame according to the initial projection parameters to obtain the actual projection parameters of the current projection frame, and to obtain the actual error increment of the projection parameters corresponding to the current projection frame based on the actual projection parameters. The determination module is used to determine the projection parameter compensation amount corresponding to the current projection frame based on the predicted drift direction of the projection parameters and the comparison result between the difference between the predicted error increment of the projection parameters and the actual error increment of the projection parameters and a preset error increment threshold. The iteration module is used to update the initial projection parameters of the current projection frame using the projection parameter compensation amount of the current projection frame, and use the updated initial projection parameters as the initial projection parameters of the next projection frame of the current projection frame, and use the next projection frame of the current projection frame as the new current projection frame, and return to execute the steps of obtaining the projection parameter prediction drift direction and projection parameter prediction error increment associated with the current projection frame.

10. A vehicle comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images