A vehicle vision sensor calibration method and system

Abstract

The present application relates to the technical field of vehicle visual sensor calibration, and discloses a vehicle visual sensor calibration method and system, which can accurately determine the current environment type by acquiring vehicle position information, preset environment information and visual perception information. When the environment type is a specific misleading environment type, the system can monitor and record the deviation information between the visual perception result and the non-visual information, effectively identify and quantify the influence of the misleading environment on the visual sensor calibration, and avoid accumulating systematic parameter deviation in uncertain environments.

Description

Technical Field

[0001] This invention relates to the field of vehicle vision sensor calibration technology, and in particular to a vehicle vision sensor calibration method and system. Background Technology

[0002] In daily driving, onboard vision sensors are a crucial component for intelligent vehicles to perceive their environment. However, these sensors may experience geometric distortions and inaccurate positioning in image acquisition when faced with complex conditions such as temperature changes, mechanical vibrations, and road bumps. Existing calibration methods often require fixed, ideal environments, making it difficult to adapt to the various dynamic changes encountered by vehicles in the real world. This leads to real-time drift of sensor parameters, affecting perception accuracy.

[0003] Especially when heavy-duty intelligent logistics trucks travel for extended periods in complex environments such as highway reconstruction and expansion construction areas, which contain numerous irregular temporary road markings and construction obstacles with fixed deviations, the visual sensor dynamic calibration system is prone to accumulating systematic parameter biases by continuously using these misleading features as references. For example, the system might misjudge a seemingly continuous temporary yellow road marking as the boundary of the current lane, or identify the edge of a row of neatly arranged guardrails as a stable feature suitable for calibration. Because these "references" themselves contain uncertainties and potential geometric biases, the confidence level of the calibration procedure decreases significantly during parameter estimation, gradually accumulating systematic calibration errors.

[0004] Furthermore, when a truck abruptly switches from this misleading environment to a normal environment with clear, standard visual features, accumulated systematic calibration biases can cause a significant discrepancy between the current parameters of the visual sensors and the characteristics of the actual environment. Dynamic calibration systems struggle to quickly determine whether the current perception error stems from a momentary, large physical displacement of the sensor itself or from systematic calibration biases accumulated during its operation within the construction area. If the system attempts to attribute this significant difference entirely to a momentary physical change in sensor parameters and makes a large-scale correction, it may push already skewed parameters further in the wrong direction, causing the calibration results to diverge rapidly. Conversely, if the system attempts to correct this difference slowly and gradually, the driver assistance functions will make decisions based on severely erroneous perceptual information for an extended period until the correction is complete, posing a direct threat to driving safety.

[0005] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0006] This invention provides a vehicle vision sensor calibration method and system, aiming to solve the technical problems of difficult calibration of on-board vision sensors in complex dynamic environments of intelligent vehicles, easy accumulation of systematic parameter deviations, and difficulty in quick and accurate correction of calibration system when switching from misleading environment to standard environment, which affects driving safety.

[0007] The technical solution of this application is as follows: In a first aspect, this application discloses a vehicle vision sensor calibration method, including: Acquire vehicle location information, preset environmental information, and visual perception information; Determine the current environment type based on location information, preset environment information, and visual perception information; When the environment type is a specific misleading environment type, the deviation information between visual perception results and non-visual information is monitored and recorded; When the environment type is switched from a specific misleading environment type to a standard environment type, a fast calibration correction is triggered; The current parameters of the vision sensor are compared with the ideal parameters calculated based on standard visual and non-visual information. Based on the comparison results and deviation information, the vision sensor parameters are quickly corrected and the normal calibration mode is restored.

[0008] This technical solution effectively identifies and addresses calibration challenges for vehicles in specific misleading environments. By recording deviation information and triggering rapid calibration correction in conjunction with environmental switching, it solves the problems of easy sensor parameter drift and limited calibration accuracy in existing technologies, significantly improving the perception accuracy and driving safety of vehicles in complex environments.

[0009] Furthermore, in the above-mentioned vehicle vision sensor calibration method, the current parameters of the vision sensor are compared with the ideal parameters calculated based on standard visual and non-visual information. Based on the comparison results and deviation information, the vision sensor parameters are quickly corrected, including: The contrast, edge integrity, continuity, and degree of matching with standard patterns of visual features are obtained to generate visual feature quality scores. Monitor the purity of the output signals from the inertial measurement unit and the high-precision positioning system, and generate a purity score for the non-visual sensors; A comprehensive information quality score is generated by fusing visual feature quality scores and non-visual sensor purity scores. Based on the comprehensive information quality score, the calibration correction factor is calculated, and the update of calibration parameters in the vision sensor calibration algorithm is adjusted accordingly. Based on the deviation information, the total deviation and deviation information between the current parameters of the vision sensor and the ideal parameters calculated based on standard visual and non-visual information are compared. Based on the comparison results, systematic deviations are eliminated or conservative corrections are made.

[0010] Through this technical solution, this application can achieve accurate and rapid correction of visual sensor parameters by comprehensively evaluating the quality of visual and non-visual information and combining historical deviation information. This effectively avoids calibration errors caused by insufficient information from a single source in complex environments, and improves the accuracy and robustness of the correction.

[0011] In some preferred embodiments, when the environment type is a specific misleading environment type, the deviation information between visual perception results and non-visual information is monitored and recorded, including: Continuously acquire non-visual information, evaluate the purity of the non-visual information, and generate a non-visual information purity score; When the purity score of non-visual information is lower than a preset threshold, non-visual information drift detection is initiated. Analyze the changing trend of non-visual information within a preset time period to determine whether there is drift in the non-visual information; When a drift in non-visual information is detected, the visual information-assisted verification mechanism is activated. After the non-visual information is corrected, the deviation between the visual perception result and the non-visual information is monitored and recorded.

[0012] Through this technical solution, this application can promptly identify and correct potential problems with non-visual information by evaluating the purity of non-visual information and detecting drift, and use visual information for auxiliary verification to ensure that the deviation information recorded in misleading environments is more accurate and reliable, providing a solid foundation for subsequent rapid calibration and correction.

[0013] Furthermore, when the environment type is switched from a specific misleading environment type to a standard environment type, a fast calibration correction is triggered, including: Continuously acquire visual and non-visual information; Analyze whether there are mixed features in visual perception information and evaluate the stability of non-visual information in the switching region; Acquire the clarity, completeness, and continuity of standard features in visual perception information, as well as the stability and accuracy of non-visual information, and generate environment switching confidence scores. When the confidence level of the environment switch reaches the preset threshold, it is determined that the current area is in the transition zone, and the progressive calibration correction mode is started in the transition zone. Based on the confidence level of environmental switching, dynamically adjust the weights of visual and non-visual information in the calibration parameter updates; When the confidence level of the environment switch is consistently higher than the preset threshold and the non-visual information remains stable, it is determined that the system has fully entered the standard environment type and a rapid calibration correction is triggered.

[0014] Through this technical solution, this application can assess the confidence level of environment switching in multiple dimensions, adopt progressive calibration correction in the transition area, and dynamically adjust the weight of visual and non-visual information. This effectively avoids correction divergence or delay caused by the calibration system's inability to quickly judge during environment switching, and achieves smooth and accurate calibration mode switching.

[0015] Building upon the above, this application further proposes dynamically adjusting the weights of visual and non-visual information in calibration parameter updates based on the confidence level of environmental switching, including: Continuously acquire visual and non-visual information; The quality of visual and non-visual information is assessed, and corresponding instantaneous quality scores are generated. Time series smoothing is applied to the instantaneous quality score to obtain a stable quality trend. Based on the steady quality trend and the confidence level of environmental switching, calculate the weights of visual and non-visual information in the calibration parameter update; The update of calibration parameters in the visual sensor calibration algorithm is adjusted according to the weights.

[0016] Through this technical solution, this application can smooth the instantaneous quality scores of visual and non-visual information and, in combination with the confidence level of environmental switching, dynamically and finely adjust the weights of the two types of information in the calibration, thereby achieving more stable and reliable calibration parameter updates during environmental switching.

[0017] As a technical improvement, the instantaneous quality score is smoothed over time to obtain a stable quality trend, including: Frequency analysis is performed on the instantaneous quality score to identify the presence of low-frequency, persistent interference components; When low-frequency, persistent interference components are identified, the parameters of the smoothing filter are adjusted according to the frequency characteristics of the interference components to smooth the instantaneous quality score. By combining vehicle motion status information, the smoothing results are corrected to obtain a stable quality trend.

[0018] Through this technical solution, this application can identify and eliminate low-frequency interference in instantaneous quality scores through frequency analysis, and make corrections in combination with vehicle motion state information to ensure that the quality trend after smoothing is more realistic and reliable, thereby further improving the accuracy and stability of calibration parameter updates.

[0019] In one implementation, when the confidence level for environment switching remains above a preset threshold and non-visual information remains stable, it is determined that the system has fully entered a standard environment type, and a rapid calibration correction is triggered, including: Continuously acquire visual and non-visual information; Evaluate the degree of matching of standard features in visual perception information and generate a visual feature matching score; Evaluate the stationarity of non-visual information and generate a stationarity score for non-visual information. By fusing visual feature matching scores and non-visual information stationary scores, a comprehensive environmental reliability score is generated, and rapid calibration correction is triggered based on the comprehensive environmental reliability score.

[0020] Through this technical solution, this application can generate a comprehensive environmental reliability score by fusing visual feature matching scores and non-visual information stability scores, thereby more comprehensively and accurately determining whether the environment has fully entered the standard environment type and triggering rapid calibration correction in a timely manner to ensure the timeliness and effectiveness of the calibration process.

[0021] To improve the solution, rapid calibration correction is triggered based on the comprehensive environmental reliability score, including: Monitor the overall environmental reliability score and compare it with a preset first reliability threshold; When the overall environmental reliability score is consistently higher than the first reliability threshold and no mixed features are detected in the visual perception information, the current environment is judged to have standard features. Start the standard feature continuous verification timer; During the operation of the standard feature continuous verification timer, the visual feature matching score and the non-visual information stability score are continuously monitored, and the overall environmental reliability score is ensured to remain above the preset second reliability threshold. When the standard feature persistence verification timer reaches the preset duration and the comprehensive environmental reliability score is always higher than the second reliability threshold, it is determined that the standard environment type has been fully entered. Trigger a fast calibration correction.

[0022] Through this technical solution, this application can ensure higher confidence and consistency in the judgment of standard environmental characteristics before triggering rapid calibration correction by introducing a continuous verification timer and multiple reliability thresholds. This effectively avoids calibration errors caused by instantaneous fluctuations or misjudgments, and further improves the reliability of calibration.

[0023] As a further improvement, during the operation of the standard feature persistence verification timer, the visual feature matching score and the non-visual information stationary score are continuously monitored, and the overall environmental reliability score is ensured to remain above a preset second reliability threshold, including: Continuously acquire visual feature matching scores and non-visual information stationary scores; Time series analysis was performed on visual feature matching scores and stationary scores of non-visual information to identify whether there were transient fluctuations; When transient fluctuations are detected, a short-term smoothing process is applied to the visual feature matching score and the non-visual information stable score to eliminate the impact of transient fluctuations. The overall environmental reliability score is recalculated based on the smoothed visual feature matching score and the non-visual information stationary score. Compare the recalculated comprehensive environmental reliability score with the preset second reliability threshold; When the recalculated overall environmental reliability score remains above the second reliability threshold, the standard feature persistence verification timer continues to run.

[0024] Through this technical solution, this application can effectively eliminate the impact of instantaneous fluctuations on the comprehensive environmental reliability score by performing time series analysis and short-term smoothing on visual and non-visual scores, ensuring that the judgment of environmental reliability is more stable and accurate during continuous verification, thereby improving the triggering accuracy of rapid calibration correction.

[0025] Secondly, this application also discloses a vehicle vision sensor calibration system, comprising: The input terminal is used to acquire vehicle location information, preset environmental information, and visual perception information. The trigger is used to determine the current environment type based on location information, preset environment information, and visual perception information; when the environment type is a specific misleading environment type, it monitors and records the deviation information between the visual perception result and non-visual information; when the environment type switches from the specific misleading environment type to the standard environment type, it triggers a fast calibration correction. The correction end is used to compare the current parameters of the vision sensor with the ideal parameters calculated based on standard visual and non-visual information. Based on the comparison results and deviation information, the vision sensor parameters are quickly corrected and the normal calibration mode is restored.

[0026] Through this technical solution, this application provides a system capable of implementing the above calibration method. Through modular design, the system can efficiently acquire environmental information, determine the environmental type, record deviation information, and trigger rapid calibration correction when the environment changes, thereby providing hardware and software support for the stable operation of vehicle vision sensors and solving the problems of low calibration efficiency and poor accuracy of existing systems in complex dynamic environments.

[0027] Beneficial effects This application discloses a vehicle vision sensor calibration method and system. By acquiring vehicle location information, preset environmental information, and visual perception information, it can accurately determine the current environment type. When the environment type is a specific misleading environment type, the system can monitor and record the deviation information between the visual perception results and non-visual information, effectively identifying and quantifying the impact of the misleading environment on vision sensor calibration, and avoiding the accumulation of systematic parameter deviations in uncertain environments. When the environment type switches from the specific misleading environment type to the standard environment type, the system can promptly trigger rapid calibration correction, compare the current parameters of the vision sensor with the ideal parameters calculated based on standard visual and non-visual information, and quickly correct the vision sensor parameters and restore the normal calibration mode based on the comparison results and recorded deviation information. This method effectively solves the problems of real-time drift of sensor parameters, limited calibration accuracy, and difficulty in rapid and accurate correction when the environment changes instantaneously in the prior art, avoiding the risk of divergent calibration results or the risk of the assisted driving function making decisions based on erroneous perception information for a long time. Through this technical solution, this application can significantly improve the perception accuracy and driving safety of vehicles in complex dynamic environments, providing a strong guarantee for the reliable operation of intelligent vehicles. Attached Figure Description

[0028] Figure 1 This is a flowchart of a vehicle vision sensor calibration method provided in an embodiment of the present invention; Figure 2 This is a flowchart of a method for rapidly correcting visual sensor parameters provided in an embodiment of the present invention; Figure 3 This is a flowchart of a method for monitoring and recording the deviation between visual perception results and non-visual information provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a vehicle vision sensor calibration system provided in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Reference Figure 1 , Figure 1 This is a flowchart of a vehicle vision sensor calibration method provided in an embodiment of the present invention, including: S11, acquire vehicle location information, preset environmental information and visual perception information; S12, determine the current environment type based on the location information, the preset environment information, and the visual perception information; S13, when the environment type is a specific misleading environment type, the deviation information between the visual perception result and the non-visual information is monitored and recorded; S14, when the environment type is switched from the specific misleading environment type to the standard environment type, a fast calibration correction is triggered; S15, compare the current parameters of the vision sensor with the ideal parameters calculated based on standard visual information and the non-visual information, and quickly correct the vision sensor parameters and restore the normal calibration mode according to the comparison results and the deviation information.

[0031] This application effectively solves the problems of poor adaptability, low calibration accuracy, and potential divergence or slow response of traditional calibration methods in complex dynamic environments by introducing environmental type judgment, deviation information recording under specific misleading environments, and a rapid calibration correction mechanism when switching environments. It significantly improves the perception accuracy and driving safety of intelligent vehicle vision sensors.

[0032] To better understand the vehicle vision sensor calibration method proposed in this application, the following will elaborate on some key terms and implementation environments involved.

[0033] "Vehicle location information" refers to the precise location data of a vehicle in a global coordinate system, usually provided by a high-precision positioning system (such as GNSS / RTK), including longitude, latitude, altitude, and vehicle attitude (such as heading angle, pitch angle, and roll angle). This information is the basis for determining the type of environment in which the vehicle is located.

[0034] "Preset environmental information" refers to pre-stored environmental feature data related to a specific geographical area, such as construction areas, tunnels, bridges, urban roads, and highways marked in high-precision map data. This information can help the system determine whether the current environment belongs to a specific misleading environmental type.

[0035] "Visual perception information" refers to environmental feature information obtained from image or video data collected by vehicle-mounted vision sensors (such as cameras) after image processing and analysis by computer vision algorithms, such as lane lines, traffic signs, obstacles, and other vehicles.

[0036] "Non-visual information" refers to environmental perception data or vehicle motion status data acquired by other onboard sensors besides visual sensors (such as millimeter-wave radar, lidar, inertial measurement unit, ultrasonic sensors, etc.). This information is usually used to supplement or verify visual information.

[0037] "Specific misleading environment types" refer to environments containing numerous non-standard, unstable, or systematically biased visual features, such as highway reconstruction and expansion construction areas, areas with sudden changes in light inside tunnels, and weather conditions like heavy fog or heavy rainfall. In these environments, relying solely on visual information for calibration can easily introduce errors.

[0038] "Standard environment type" refers to an environment with clear, stable, and standardized visual characteristics, such as a standard highway or urban main road under clear weather conditions. In these environments, visual sensors can obtain high-quality perception data, which is beneficial for accurate calibration.

[0039] "Bias information" refers to the discrepancy between visual perception and non-visual information in a specific, misleading environment. This discrepancy may be due to visual sensor parameter drift, misleading environmental features, or both. Recording this bias information helps in compensation during subsequent calibration.

[0040] "Rapid calibration correction" refers to the system's ability to quickly adjust the visual sensor parameters to adapt to the new environmental characteristics and eliminate previously accumulated biases when the environment type switches from a specific misleading environment to a standard environment.

[0041] The implementation environment of this application is typically intelligent driving vehicles, especially heavy-duty intelligent logistics trucks, which operate for long periods of time in complex road environments, requiring extremely high calibration accuracy and robustness of vision sensors.

[0042] The core of the vehicle vision sensor calibration method proposed in this application lies in intelligently determining the type of environment and adopting differentiated calibration strategies for different environments, thereby improving the accuracy and robustness of calibration.

[0043] First, in the steps of "acquiring vehicle location information, preset environmental information, and visual perception information," the vehicle continuously obtains necessary information from multiple data sources. For example, vehicle location information can be acquired in real time through an onboard high-precision positioning module (such as a GPS / BeiDou RTK system), which provides precise latitude, longitude, altitude, speed, and attitude data. Preset environmental information can be queried from a pre-installed high-precision map database, which may contain geospatial information such as road types, construction zone markers, and tunnel locations. Visual perception information is collected in real time by onboard cameras and preliminarily analyzed by an image processing unit to extract visual features such as lane lines, traffic signs, and obstacles. This information forms the basis for subsequent environmental assessment and calibration.

[0044] Secondly, in the step of "determining the current environment type based on the location information, the preset environment information, and the visual perception information," the system comprehensively utilizes the information acquired above to identify the vehicle's environment. For example, the system can first determine whether the vehicle has entered a construction area or tunnel marked on a high-precision map based on the vehicle's location information and the preset environment information. Simultaneously, by analyzing visual perception information, such as detecting the clarity and integrity of lane lines, and the presence of numerous temporary markings or irregular obstacles, the system can further verify or correct the environment judgment based on map information. For instance, if the visual information shows blurred lane lines or numerous construction barriers, even if the map information does not explicitly mark it as a construction area, the system may still determine that the current environment is a specific misleading type.

[0045] Furthermore, when the environment type is a specific misleading environment type, the system monitors and records the deviation information between visual perception results and non-visual information. In specific misleading environments, because visual features may be inaccurate or misleading, the system relies more heavily on non-visual information as a reference. For example, in a construction area, visual sensors may misidentify temporary markings as lane lines, causing a deviation between the visual perception results and the actual lane position. In this case, the system continuously acquires non-visual information, such as vehicle trajectory data from an inertial measurement unit (IMU) and obstacle distance and angle information from millimeter-wave radar or lidar. By comparing the visual perception results (e.g., the visually recognized lane line position) with non-visual information (e.g., the vehicle's offset relative to the lane center calculated by the IMU), the system can calculate and record the deviation information between the two. This deviation information is stored for use in subsequent calibration and correction. For example, if the vision system continuously identifies a temporary marking 10 centimeters off the actual lane center as the lane center, then this 10-centimeter deviation will be recorded.

[0046] Next, when the environment type switches from the specific misleading environment type to the standard environment type, a rapid calibration correction is triggered. The system detects the environment type switch when the vehicle leaves the specific misleading environment (e.g., exiting a construction area) and enters a standard environment (e.g., entering a clear, well-maintained highway section). This switch can be determined based on a comparison of location information with preset environment information, or by analyzing the appearance of standard features in visual perception information and the stability of non-visual information. Once the environment type switch is confirmed, the system immediately triggers the rapid calibration correction mechanism, rather than waiting for a regular, periodic calibration process. This rapid response mechanism aims to quickly eliminate systematic biases accumulated in misleading environments.

[0047] Finally, in the step of "comparing the current parameters of the visual sensor with the ideal parameters calculated based on standard visual information and the aforementioned non-visual information, and quickly correcting the visual sensor parameters and restoring the normal calibration mode according to the comparison results and the deviation information," the system performs specific calibration operations. In a standard environment, the system can acquire high-quality standard visual information and reliable non-visual information. Based on this information, the system can calculate the "ideal parameters" of the visual sensor in the current standard environment, that is, the parameters that the sensor should possess that can accurately reflect the real environment. Then, the system compares the current parameters of the visual sensor with these ideal parameters to obtain a total deviation. When making corrections, the system comprehensively considers this total deviation as well as the deviation information previously recorded in specific misleading environments. For example, if the previously recorded deviation information shows that the visual system has a persistent systematic deviation of 10 cm to the left, then during rapid correction, the system will prioritize eliminating this systematic deviation, rather than simply attributing all differences between the current parameters and the ideal parameters to instantaneous physical changes. In this way, the system can more accurately and quickly adjust the visual sensor parameters to their optimal state, and then return to the regular, periodic calibration mode to maintain the long-term accuracy of the sensor.

[0048] The vehicle vision sensor calibration method proposed in this application effectively solves many problems existing in the prior art by introducing environmental type judgment, recording deviation information under specific misleading environments, and a rapid calibration correction mechanism when the environment changes. Traditional calibration methods often need to be carried out in fixed, ideal environments, which are difficult to adapt to the various dynamic changes encountered by vehicles in the real world, resulting in real-time drift of sensor parameters and affecting perception accuracy. Especially when heavy-duty intelligent logistics trucks travel for long periods in complex environments such as highway reconstruction and expansion construction areas with a large number of non-standard temporary markings and construction obstacles with fixed deviations, the dynamic calibration system of vision sensors is prone to accumulating systematic parameter deviations by continuously using these misleading features as references.

[0049] The core innovation of this application lies in its approach: instead of treating all environments in a blanket manner, it intelligently distinguishes between "specific misleading environment types" and "standard environment types." In specific misleading environments, the system no longer blindly calibrates but instead "monitors and records the deviation information between visual perception results and non-visual information." This strategy avoids accumulating erroneous calibration parameters in unreliable environments, instead storing these deviations as "empirical data." For example, in a construction area, the system records the persistent deviation of the vision system relative to radar or IMU, rather than attempting immediate correction.

[0050] When the environment switches from a specific misleading environment to a standard environment, this application can "trigger rapid calibration correction." This contrasts sharply with the potential for divergence or slow response in existing calibration systems. In conventional solutions, when a vehicle suddenly enters a standard environment from a misleading environment, due to previously accumulated biases, there may be significant discrepancies between the visual sensor parameters and the real environment, making it difficult for the system to quickly identify and effectively correct these discrepancies. This application, however, utilizes previously recorded bias information to "compare the current parameters of the visual sensor with the ideal parameters calculated based on standard visual information and the non-visual information during rapid calibration correction, and based on the comparison results and the bias information, quickly correct the visual sensor parameters and restore the normal calibration mode." This means that during correction, the system not only considers the ideal parameters in the current environment but also incorporates the bias information accumulated in the misleading environment, thereby more accurately eliminating systematic biases and avoiding erroneous corrections caused by attributing all differences to instantaneous physical changes.

[0051] For example, suppose in a construction zone, a vision sensor continuously identifies a temporary lane marking 10 centimeters off the actual lane center as the lane center and records this 10-centimeter systematic deviation. When the vehicle leaves the construction zone and enters a standard highway, if the vision sensor still shows a 10-centimeter deviation to the left, the calibration method in this application will prioritize using the previously recorded 10-centimeter deviation information for correction, rather than simply attributing the current 10-centimeter deviation entirely to the instantaneous physical displacement of the sensor. This correction method based on historical deviation information makes the calibration process more stable, faster, and more accurate, significantly improving the perception robustness and driving safety of intelligent vehicles in complex dynamic environments.

[0052] In some embodiments described above, a scheme for triggering rapid calibration correction when the environment type switches from a specific misleading environment type to a standard environment type is proposed. However, in its implementation, the step of rapidly correcting the visual sensor parameters in the basic scheme may not fully consider the real-time quality and reliability of visual perception information and non-visual information. This may lead to an impact on the accuracy and stability of calibration correction when information quality fluctuates significantly, and may even introduce new errors. If the above problems are not addressed, the calibration results may be inaccurate, thereby affecting the vehicle's perception and decision-making capabilities. To address this, this application further proposes a more refined rapid calibration correction method, which improves the robustness and accuracy of calibration by comprehensively evaluating information quality and dynamically adjusting the correction strategy.

[0053] In this regard, refer to Figure 2 , Figure 2 This is a flowchart of a method for rapidly correcting visual sensor parameters according to an embodiment of the present invention. S15 includes: S151, acquire the contrast, edge integrity, continuity and matching degree with standard patterns of visual features, and generate visual feature quality scores; S152 monitors the purity of the output signals from the inertial measurement unit and the high-precision positioning system, and generates a purity score for non-visual sensors. S153, The visual feature quality score and the non-visual sensor purity score are fused to generate a comprehensive information quality score; S154, Based on the comprehensive information quality score, calculate the calibration correction factor and adjust the update of the calibration parameters in the visual sensor calibration algorithm; S155, based on the deviation information, compare the total deviation between the current parameters of the visual sensor and the ideal parameters calculated based on the standard visual information and the non-visual information, and the deviation information, and based on the comparison result, eliminate systematic deviations or make conservative corrections.

[0054] Specifically, the acquisition of visual features' contrast, edge integrity, continuity, and matching degree with standard patterns aims to quantify the reliability of visual perception information. Contrast refers to the degree of brightness difference between different regions in an image; high contrast usually indicates sharper features. Edge integrity refers to the sharpness and continuity of object edges in an image; broken or blurred edges reduce feature reliability. Continuity focuses on the stability of visual features over time, such as the appearance of the same object in consecutive frames. Matching degree with standard patterns refers to the similarity between the current visual feature and a pre-set ideal feature template acquired under standard conditions. By comprehensively evaluating these indicators, a visual feature quality score can be generated, reflecting the usability and reliability of the current visual perception information.

[0055] Simultaneously, the purity of the output signals from the inertial measurement unit (IMU) and high-precision positioning system (HPS) is monitored, and a non-visual sensor purity score is generated to assess the quality of non-visual information. Inertial measurement units (IMUs) and high-precision positioning systems (such as GNSS RTK) are crucial components of a vehicle's perception system, and the purity of their output signals directly impacts the accuracy of vehicle attitude and position information. Purity assessment can include analysis of signal noise levels, drift trends, data integrity, and deviations from expected models. For example, purity decreases when signals are interfered with or obstructed. By quantifying the purity of these non-visual sensor signals, a non-visual sensor purity score can be generated, characterizing the reliability of non-visual information.

[0056] Furthermore, the visual feature quality scores and the non-visual sensor purity scores are fused to generate a comprehensive information quality score. This fusion process can employ weighted averaging, Bayesian fusion, or other multi-sensor fusion algorithms, with the aim of integrating sensor quality assessment results from different modalities into a unified and comprehensive information quality index. This comprehensive information quality score can more comprehensively reflect the overall reliability of all available perceptual information in the current environment, providing a more accurate confidence basis for subsequent calibration and correction.

[0057] Based on this, a calibration correction factor is calculated according to the comprehensive information quality score, and the update of calibration parameters in the visual sensor calibration algorithm is adjusted accordingly. The calibration correction factor is a dynamically adjusted parameter, and its value is positively correlated with the comprehensive information quality score. When the comprehensive information quality score is high, it indicates that the current perceived information is highly reliable, and the calibration correction factor can be larger, allowing the calibration algorithm to update parameters more aggressively. Conversely, when the score is low, the correction factor should be smaller, making the update of calibration parameters more conservative to avoid introducing errors caused by low-quality information. In this way, the calibration process can adaptively respond to changes in the environment and sensor state.

[0058] Finally, based on the deviation information, the total deviation between the current parameters of the vision sensor and the ideal parameters calculated based on the standard visual information and the non-visual information is compared with the deviation information. Based on the comparison results, systematic deviations are eliminated or conservative corrections are made. The total deviation refers to the overall difference between the current parameters of the vision sensor and the ideal parameters. The deviation information is the systematic difference between visual perception results and non-visual information pre-monitored and recorded by the vehicle in a specific misleading environment. By comparing the current total deviation with historical deviation information, systematic deviations caused by specific environments (such as tunnels, strong light, etc.) can be identified. If the comparison results show that the current deviation highly matches historical systematic deviations, these systematic deviations can be eliminated in a targeted manner, thereby quickly adjusting the sensor parameters to be closer to the ideal state. If the comparison results show that the deviation has high uncertainty or does not match historical deviations, a conservative correction strategy can be adopted, i.e., updating the calibration parameters in smaller steps or in a more cautious manner to avoid over-correction or introducing new errors.

[0059] This application addresses the issue of insufficient accuracy and stability in rapid calibration correction due to information quality fluctuations in basic solutions by introducing a quantitative assessment of the quality of both visual and non-visual information. Through this technical solution, the application effectively improves the accuracy and stability of rapid calibration correction for vehicle vision sensors. Specifically, by performing a refined quality assessment of visual features and non-visual sensor signals and integrating them into a comprehensive information quality score, the calibration system can intelligently adjust its correction strategy based on the reliability of the current data. This avoids blind or excessive correction when data quality is poor, thereby reducing the introduction of calibration errors. Simultaneously, the introduction of a calibration correction strength factor enables adaptive adjustment of calibration parameter updates, making the calibration process more flexible and efficient. Furthermore, by incorporating deviation information under specific misleading environments, systematic biases can be eliminated in a targeted manner, or conservative corrections can be made when necessary, further ensuring the accuracy and reliability of the calibration results, ultimately improving the vehicle's perception capabilities and safety after switching to complex environments.

[0060] In some preferred embodiments, an autonomous vehicle is assumed to exit a dimly lit, highly reflective road segment (a specific misleading environment type) and enter a well-lit, clearly marked open road segment (a standard environment type). Before exiting the misleading environment, the system has monitored and recorded the deviation between visual perception results and non-visual information. When the vehicle enters the standard environment type, a rapid calibration correction is triggered. At this point, the system first acquires visual perception information and analyzes the contrast, edge integrity, continuity, and matching degree with standard templates of visual features such as road markings and traffic signs, thereby generating a visual feature quality score. Simultaneously, the purity of the output signals from the vehicle's inertial measurement unit and high-precision positioning system is monitored, for example, by checking the number of satellites in the GPS signal and the confidence level of HD map matching, to generate a non-visual sensor purity score. Subsequently, these two scores are fused to obtain a comprehensive information quality score.

[0061] For example, upon entering a standard environment, visual features may still have residual blur, and non-visual signals may experience brief fluctuations, resulting in a generally low to moderate information quality score. The system calculates a moderate calibration correction factor based on this score and adjusts the parameter updates in the vision sensor calibration algorithm in a relatively conservative manner. As the vehicle travels in a standard environment, visual features become clearer, and non-visual signals stabilize, gradually increasing the overall information quality score. The calibration correction factor also increases accordingly, making the calibration parameter updates more proactive. During this process, the system compares the total deviation between the current vision sensor parameters and the ideal parameters, incorporating deviation information previously recorded in misleading environments. If the current total deviation contains systematic deviations similar to historical records (e.g., fixed-angle deviations due to specific reflections), the system prioritizes eliminating these deviations. If the current deviation exhibits high randomness or does not match historical deviations, conservative corrections are applied to ensure the accuracy and stability of the calibration.

[0062] In some embodiments described above in this application, when the environment type is a specific misleading environment type, deviation information between visual perception results and non-visual information is monitored and recorded. However, during implementation, if the non-visual information itself has uncertainties or drift, the recorded deviation information may not accurately reflect the true error of the visual sensor, which may lead to insufficient accuracy in subsequent calibration corrections or even the introduction of new errors.

[0063] In this regard, refer to Figure 3 , Figure 3 This is a flowchart of a method for monitoring and recording the deviation between visual perception results and non-visual information provided in an embodiment of the present invention. S13 includes: S131, continuously acquire the non-visual information, evaluate the purity of the non-visual information, and generate a non-visual information purity score. S132, when the purity score of the non-visual information is lower than a preset threshold, non-visual information drift detection is initiated; S133, Analyze the change trend of the non-visual information within a preset time period, and determine whether the non-visual information has drifted; S134, when a drift in the non-visual information is detected, the visual information-assisted verification mechanism is activated; S135, after the non-visual information is corrected, monitor and record the deviation information between the visual perception result and the non-visual information.

[0064] Specifically, continuously acquiring non-visual information refers to the continuous reception of data streams from non-visual sensors such as inertial measurement units (IMUs), global positioning systems (GPS), wheel speed sensors, radar, or lidar during vehicle operation. These data streams can include information such as the vehicle's attitude, speed, position, and acceleration. The purpose of evaluating the purity of non-visual information and generating a non-visual information purity score is to quantify the reliability and stability of the non-visual information. For example, purity evaluation can be calculated based on signal-to-noise ratio, data consistency, sensor health status, and comparison results with redundant sensor data. The non-visual information purity score can be a value between 0 and 1, with a higher score indicating purer and more reliable information.

[0065] The preset threshold is a value determined based on system design requirements and actual application scenarios, used to define the acceptable range of non-visual information purity. When the non-visual information purity score is lower than the preset threshold, it indicates that the non-visual information may be abnormal or unreliable, requiring further detection and processing.

[0066] In practical applications, initiating non-visual information drift detection means that when the purity of non-visual information is insufficient, the system will activate a dedicated algorithm module to perform a deeper analysis of the non-visual information. This involves analyzing the changing trends of the non-visual information over a preset time period to determine if drift exists. For example, sliding window averaging, Kalman filtering, or statistical methods (such as mean, variance, and slope changes) can be used to monitor changes in non-visual information over time. Drift typically manifests as a continuous, unidirectional, or periodic deviation in non-visual information in the absence of any real external changes. The preset time can be set based on the vehicle's motion characteristics and sensor response time, for example, from a few seconds to tens of seconds.

[0067] When drift in non-visual information is detected, a visual information-assisted verification mechanism is activated. This mechanism uses visual information as an independent reference source to cross-validate the accuracy of the non-visual information. For example, vehicle motion and position can be independently estimated using visual odometry or map-based SLAM and compared with drifting non-visual information. If a significant inconsistency exists between the visual and non-visual information, the drift in the non-visual information is further confirmed.

[0068] After the non-visual information correction is completed, the deviation between the visual perception result and the non-visual information is monitored and recorded. Non-visual information correction may include filtering, correcting, or compensating for drifting non-visual information by fusing other reliable information sources. After correction, ensure that the non-visual information is restored to a reliable state, and then record the deviation between the visual perception result and the corrected non-visual information to obtain more accurate deviation data.

[0069] This application addresses the problem of inaccurate deviation information recording caused by uncertainties or drift in non-visual information under specific misleading environmental conditions by introducing a non-visual information purity assessment and drift detection mechanism. Through this technical solution, the application effectively avoids recording erroneous visual sensor deviation information when non-visual information is drifted or impure, significantly improving the accuracy and reliability of deviation information recording. This allows subsequent rapid calibration correction to be based on more realistic and reliable deviation data, thereby improving the accuracy and robustness of visual sensor calibration. Especially under specific misleading environmental conditions, it effectively prevents the accumulation of calibration errors caused by unreliable non-visual information, ensuring the long-term stable operation of the vehicle perception system.

[0070] In some preferred embodiments, assume an autonomous vehicle is traversing a long tunnel. At the tunnel entrance, GPS signals may begin to weaken, leading to a decrease in the purity of non-visual information (such as vehicle position). The system continuously acquires non-visual information from sensors such as the IMU and wheel speedometer, and calculates a purity score for this non-visual information. When the GPS signal is completely lost and the purity score falls below a preset threshold, the system initiates non-visual information drift detection. At this point, the IMU's integration error may begin to accumulate, causing position estimation drift. The system analyzes the trend of the IMU's output position data over the past 10 seconds. If a persistent and increasing deviation is found between the IMU and wheel speedometer data, non-visual information drift is identified. At this point, a visual information-assisted verification mechanism is activated. The vehicle's visual odometry independently calculates the vehicle's displacement in the tunnel and compares it with the non-visual information fused from the IMU and wheel speedometer. If the visual odometry shows the vehicle has traveled 50 meters in a straight line, but the non-visual information shows 55 meters, non-visual information drift is confirmed. The system uses visual odometry data to correct non-visual information, for example, by fusing visual and non-visual information using a Kalman filter to correct IMU drift. Only after the non-visual information correction is complete does the system begin monitoring and recording the deviation between the visual perception results (e.g., lane line positions detected by the visual sensor) and the corrected non-visual information (e.g., the corrected actual vehicle position). This accurate deviation information is used for subsequent rapid calibration correction, ensuring that the visual sensor can quickly and accurately resume calibration after regaining GPS signals at the tunnel exit.

[0071] In some embodiments described above, a rapid calibration correction is triggered when the environment type switches from a specific misleading environment type to a standard environment type. However, in practical applications, the switching of environment types is often not instantaneous, but rather involves a transition region. This region may simultaneously contain both specific misleading environment characteristics and standard environment characteristics, or non-visual information may experience brief instability during the switching process. If rapid calibration correction is triggered directly based solely on a simple environment type judgment, the calibration process may not be smooth enough, or correction may be performed before the environment is fully stable, thereby affecting the accuracy and stability of the calibration.

[0072] In this regard, this application further proposes that the steps for triggering rapid calibration correction when the environment type is switched from a specific misleading environment type to a standard environment type include: Continuously acquire visual and non-visual information; Analyze whether there are mixed features in the visual perception information, and evaluate the stability of the non-visual information in the switching region; The clarity, completeness, and continuity of standard features in the visual perception information, as well as the stability and accuracy of the non-visual information, are obtained to generate an environment switching confidence score. When the confidence level of the environment switch reaches a preset threshold, it is determined that the current area is in the transition zone, and the progressive calibration correction mode is started in the transition zone. Based on the confidence level of the environment switch, the weights of the visual information and the non-visual information in the calibration parameter update are dynamically adjusted. When the confidence level of the environment switching is consistently higher than the preset threshold and the non-visual information remains stable, it is determined that the system has fully entered the standard environment type, and a rapid calibration correction is triggered.

[0073] Specifically, continuously acquiring visual and non-visual information refers to the vehicle's visual sensors continuously collecting image or video data, while non-visual sensors such as the Inertial Measurement Unit (IMU) and Global Positioning System (GPS) continuously output their perception data. Analyzing whether the visual perception information contains mixed features involves image processing and feature extraction of the visual data to identify whether it contains features from both specific misleading environments (e.g., low light in a tunnel, or blurred images in rain or fog) and standard environments (e.g., clear road markings and traffic signs). Simultaneously, assessing the stability of the non-visual information in the switching region involves performing time-series analysis on non-visual sensor data (such as vehicle speed, attitude, and position) to determine whether there are abnormal fluctuations or drifts during environmental transitions.

[0074] Furthermore, the sharpness, completeness, and continuity of standard features in the visual perception information, as well as the stationarity and accuracy of the non-visual information, are obtained to generate an environment switching confidence score. The sharpness, completeness, and continuity of the standard features can be quantitatively evaluated using image processing algorithms, such as edge detection and texture analysis. The stationarity and accuracy of the non-visual information can be evaluated using statistical methods, such as analysis of variance and Kalman filter residuals. These evaluation results are fused, for example, through weighted averaging or a machine learning model, to generate a comprehensive environment switching confidence score, which reflects the degree of certainty of switching from the current environment to the standard environment.

[0075] When the environment transition confidence level reaches a preset threshold, it indicates that the vehicle may be transitioning from a misleading environment to a standard environment. Within this transition region, a progressive calibration correction mode is activated. This means that the calibration algorithm will not immediately make large corrections, but will gradually adjust the visual sensor parameters in smaller, more conservative steps. Simultaneously, based on the environment transition confidence level, the weights of visual and non-visual information in the calibration parameter updates are dynamically adjusted. For example, when the confidence level is low, the weight of non-visual information may be higher to ensure calibration stability; as the confidence level gradually increases, the weight of visual information will increase accordingly to utilize its richer environmental details.

[0076] Finally, when the confidence level of the environment switch remains higher than the preset threshold and the non-visual information remains stable, the system determines that the vehicle has fully entered the standard environment type. At this point, the environmental conditions meet the requirements for rapid calibration correction, thereby triggering rapid calibration correction so that the visual sensor parameters can be quickly adjusted to the optimal state.

[0077] This application's solution effectively addresses the robustness issues that may exist in the basic solution's environmental switching judgment by introducing refined perception and progressive processing of the environmental switching process. Through the above technical solution, this application achieves significant optimization of the vehicle vision sensor calibration process. First, by comprehensively evaluating the mixed features and the stability of non-visual information in visual perception information, the true state of environmental type switching can be judged more accurately and robustly, avoiding calibration corrections when the environment is uncertain or unstable, thereby improving calibration reliability. Second, the introduction of a progressive calibration correction mode and a dynamic weight adjustment mechanism allows the calibration process to smoothly adapt to environmental changes, effectively reducing oscillations and instability during calibration, and ensuring the continuity and accuracy of calibration. Finally, rapid calibration correction is triggered only when the environment is completely stable and the confidence level is sufficiently high, ensuring the accuracy of the calibration results and effectively avoiding miscalibration or poor calibration results caused by inaccurate environmental judgment, thereby improving the overall performance and safety of the vehicle vision perception system.

[0078] In some preferred embodiments, a specific example is given below. Suppose an autonomous vehicle is exiting a dimly lit and wet tunnel (a specific misleading environment type) and entering a sunny and dry open highway (a standard environment type).

[0079] First, the vehicle's visual sensors continuously acquire visual images of the tunnel exit, which may include low-light conditions and reflections inside the tunnel, as well as bright light and clear road surface features at the exit, i.e., a mixture of features. Simultaneously, non-visual sensors such as the inertial measurement unit (IMU) and high-precision positioning system (GNSS) continuously provide information on the vehicle's speed, attitude, and position.

[0080] The system analyzes visual perception information to identify situations where specific misleading features such as drastic changes in light and road surface reflections coexist with standard features such as clear lane markings and traffic signs on highways. Simultaneously, it assesses the stability of non-visual information in the tunnel exit area; for example, if a vehicle encounters crosswinds or road bumps at the exit, IMU data may experience brief fluctuations.

[0081] Based on these analyses, the system acquires the clarity, completeness, and continuity of standard features (such as lane lines) in visual perception information, as well as the stability and accuracy of non-visual information (such as vehicle position and speed), and integrates this information to generate an environment transition confidence score. For example, when a vehicle is just exiting a tunnel, the confidence score may be low. As the vehicle leaves the tunnel, the standard features become clearer and the non-visual information tends to stabilize, and the confidence score gradually increases.

[0082] When the confidence level for environment switching reaches a preset threshold (e.g., 0.6), the system determines that it is currently in a transition region. At this point, the system initiates a progressive calibration correction mode, where the calibration algorithm adjusts parameters with smaller steps and a more conservative strategy. Simultaneously, the weights of visual and non-visual information in the calibration parameter updates are dynamically adjusted based on the confidence level. For example, at a low confidence level, the weight of non-visual information might be set to 0.7, and the weight of visual information to 0.3; as the confidence level increases to 0.8, the weight of both non-visual and visual information might be adjusted to 0.5 to fully utilize the gradually reliable visual information.

[0083] Finally, when the environment switching confidence level remains above a preset threshold (e.g., 0.85) and non-visual information (such as GNSS positioning accuracy and IMU attitude stability) remains stable for a period of time (e.g., 5 seconds), the system determines that the vehicle has fully entered the standard environment type. At this point, a rapid calibration correction is triggered, and the visual sensor parameters are quickly adjusted to the optimal state that matches the current standard environment, thereby ensuring that the vehicle can obtain accurate and reliable visual perception data on the highway.

[0084] In some embodiments described above, this application proposes dynamically adjusting the weights of visual and non-visual information in calibration parameter updates based on the confidence level of environmental switching. However, in its implementation, adjusting the weights solely based on the confidence level of environmental switching may not adequately address the impact of instantaneous quality fluctuations in both visual and non-visual information on the calibration results. These instantaneous fluctuations may lead to insufficient precision or stability in the weight adjustments, thereby affecting the efficiency and accuracy of rapid calibration correction.

[0085] In response, this application further proposes the method for dynamically adjusting the weights of visual and non-visual information in calibration parameter updates based on the confidence level of environmental switching. The specific steps include: Continuously acquire the visual information and the non-visual information; The visual information and the non-visual information are evaluated for quality, and corresponding instantaneous quality scores are generated. The instantaneous quality score is smoothed over time to obtain a stable quality trend; Based on the steady-state quality trend and the confidence level of the environment switching, calculate the weights of the visual information and the non-visual information in the calibration parameter update; The update of calibration parameters in the visual sensor calibration algorithm is adjusted according to the weights.

[0086] Specifically, continuously acquiring visual and non-visual information refers to the vehicle receiving image or video data from visual sensors (such as cameras) and various data from non-visual sensors (such as inertial measurement units, high-precision positioning systems, radar, and lidar) in real time during operation. This information forms the basis for subsequent quality assessment and weight calculation. Quality assessment of both visual and non-visual information, generating corresponding instantaneous quality scores, can be understood as real-time analysis of the acquired visual and non-visual data to quantify their reliability and usability. For example, quality assessment of visual information may include analysis of indicators such as image sharpness, noise level, feature point density, and adaptability to lighting conditions; quality assessment of non-visual information may include analysis of indicators such as signal purity, noise level, data integrity, and positioning accuracy. Through these assessments, an instantaneous value can be generated for each type of information, reflecting its current quality level.

[0087] In practical applications, time-series smoothing of instantaneous quality scores to obtain a stable quality trend refers to processing continuous instantaneous quality scores using time-series analysis methods to eliminate the impact of instantaneous noise or short-term fluctuations on the quality scores. For example, algorithms such as moving averages, exponential smoothing, or Kalman filtering can be used to filter instantaneous quality scores, thereby obtaining a stable trend that better reflects the true changes in information quality. The purpose is to ensure that subsequent weight calculations are based on more stable and reliable quality assessment results.

[0088] Furthermore, calculating the weights of visual and non-visual information in the calibration parameter update based on the stable quality trend and the confidence level of the environment switch means comprehensively considering the confidence level of the environment switch and the intrinsic quality trends of visual and non-visual information to determine their respective contribution ratios in the calibration algorithm. For example, when the confidence level of the environment switch is high and the quality trend of visual information is good, the weight of visual information can be appropriately increased; conversely, when the quality trend of non-visual information is more stable, the weight of non-visual information can be increased. This dynamic adjustment mechanism aims to maximize the use of the most reliable information source for calibration. Therefore, adjusting the update of calibration parameters in the visual sensor calibration algorithm according to the weights means applying the calculated weights to the visual sensor calibration algorithm to control the update rate and direction of calibration parameters (such as intrinsic and extrinsic parameters). For example, in optimization- or filtering-based calibration algorithms, weights can be used as coefficients in the loss function or gains in the state update equation, thereby affecting the convergence speed and final accuracy of the calibration parameters.

[0089] This application's solution effectively addresses the limitations of adjusting weights solely based on environment switching confidence levels by introducing quality assessments of both visual and non-visual information and time-series smoothing. Through this technical solution, the application achieves more refined and robust dynamic adjustment of the weights for updating visual sensor calibration parameters. Compared to relying solely on environment switching confidence levels for weight adjustment, this solution effectively improves the accuracy and stability of weight calculation by introducing real-time quality assessments and time-series smoothing of both visual and non-visual information. This allows the rapid calibration correction process to more effectively utilize the most reliable information sources when a vehicle switches from a specific misleading environment type to a standard environment type, significantly improving the accuracy and convergence speed of calibration parameter updates, reducing calibration errors caused by instantaneous quality fluctuations in information sources, and ultimately ensuring the perception performance and safety of the visual sensor under complex environment switching conditions.

[0090] In some preferred embodiments, a specific example is given below. Suppose a vehicle is exiting a dimly lit tunnel and entering a bright, open road. In this transitional zone, the confidence level for the environmental change gradually increases. First, the system continuously acquires data streams from the vehicle's onboard camera (visual information), inertial measurement unit (IMU), and high-precision GNSS (non-visual information). Next, the visual information is quality-assessed, analyzing factors such as brightness, contrast, sharpness, and the presence of glare or shadows, generating an instantaneous visual quality score. Simultaneously, the non-visual information is quality-assessed, monitoring factors such as the noise level of the IMU data, the number of satellites in the GNSS signal, and positioning accuracy, generating an instantaneous non-visual quality score. These instantaneous quality scores may fluctuate because the vehicle may experience brief, drastic changes in light or abrupt changes when the GNSS signal recovers instantaneously at the tunnel exit.

[0091] Therefore, the system performs time-series smoothing on these instantaneous quality scores, for example, by using a moving average filter to eliminate short-term noise effects and obtain a more stable visual and non-visual quality trend. Subsequently, the system comprehensively considers the current environment switching confidence level (e.g., gradually increasing from 0.3 to 0.8) and the stable visual and non-visual quality trends. For example, if the environment switching confidence level is high, and the stable visual quality trend indicates that image quality is rapidly improving while the non-visual quality trend remains stable, the system will calculate a higher weight for visual information and a relatively lower weight for non-visual information. Conversely, if at some moment the visual information quality trend slightly decreases due to a brief period of strong backlighting, but the non-visual information quality trend remains very stable, the system will dynamically adjust, appropriately increasing the weight of non-visual information to compensate for the temporary deficiency in visual information. Finally, based on these calculated dynamic weights, the visual sensor calibration algorithm will adjust the updates of its calibration parameters accordingly. For example, in Kalman filter-based calibration algorithms, these weights can serve as adjustment factors for the observation noise covariance matrix, allowing higher-quality information sources to contribute more to state estimation, thus achieving faster and more accurate calibration correction. In this way, even in rapidly changing transitional regions, the calibration parameters of the visual sensor can be updated stably and efficiently.

[0092] In some embodiments described above, a time-series smoothing process is proposed to obtain a stable quality trend from the instantaneous quality score, which is then used to calculate the weights of visual and non-visual information in calibration parameter updates. However, in practical applications, the instantaneous quality score may be affected by various factors, such as ambient light fluctuations and sensor noise, which may introduce low-frequency, persistent interference components. If only a general smoothing filter is used, it may not be able to effectively filter out such specific interferences, leading to a decrease in the accuracy of the stable quality trend. Furthermore, the characteristics of sensor data from a vehicle vary under different motion states (such as acceleration, deceleration, turning, or stationary). Simple smoothing may not be able to adequately adapt to these dynamic changes, thereby affecting the authenticity and reliability of the stable quality trend.

[0093] In this regard, this application further proposes the following steps for performing time series smoothing on the instantaneous quality score to obtain a stationary quality trend: Frequency analysis is performed on the instantaneous quality score to identify whether there are low-frequency, persistent interference components in the instantaneous quality score; When the low-frequency, persistent interference component is identified, the parameters of the smoothing filter are adjusted according to the frequency characteristics of the interference component to smooth the instantaneous quality score. By combining vehicle motion status information, the smoothing results are corrected to obtain a stable quality trend.

[0094] Specifically, frequency analysis of instantaneous quality scores involves transforming the instantaneous quality score from the time domain to the frequency domain using Fourier transform or other spectral analysis methods to reveal periodic or quasi-periodic components. By analyzing the spectrum, energy concentrations within specific frequency ranges can be identified. These energy concentrations may correspond to low-frequency, persistent interference components, such as signal noise caused by ambient light fluctuations (e.g., streetlight flickering) or slow drift within the sensor. The purpose of identifying these interference components is to provide a basis for subsequent targeted filtering.

[0095] When low-frequency, persistent interference components are identified, the parameters of the smoothing filter are adjusted according to the frequency characteristics of the interference components to smooth the instantaneous quality score. This means that the smoothing filter is no longer a general filter with fixed parameters, but rather it adaptively adjusts based on the detected interference frequency. For example, if interference is identified in a specific low-frequency range, the cutoff frequency and attenuation slope of a band-stop or low-pass filter can be designed or adjusted to effectively suppress interference within that frequency range while preserving as much useful information as possible in the instantaneous quality score. This adaptive parameter adjustment ensures the targeted and effective smoothing process.

[0096] In practical applications, the smoothing results are corrected by incorporating vehicle motion state information to obtain a stable quality trend. Vehicle motion state information can include vehicle speed, acceleration, angular velocity, steering angle, etc. This information reflects the dynamic environment in which the vehicle is located. For example, when a vehicle is traveling at high speed or making sharp turns, the instantaneous quality scores of visual and non-visual information may fluctuate rapidly. In this case, excessive smoothing may result in the loss of useful information. Conversely, when the vehicle is stationary or traveling at a constant speed in a straight line, the quality score should be relatively stable, and a stronger smoothing can be used. By using vehicle motion state information as a correction factor, the smoothed results can be further corrected. For example, when the vehicle's motion state changes drastically, the smoothing effect can be appropriately weakened to reflect the actual changes; when the vehicle's motion state is stable, the smoothing effect can be enhanced to better filter out noise, thereby obtaining a stable quality trend that more closely reflects reality.

[0097] This application's solution effectively addresses the limitations of traditional single-mode smoothing in complex dynamic environments by introducing frequency analysis and vehicle motion state information. Through this technical solution, the application significantly improves the accuracy and robustness of instantaneous quality score smoothing. Specifically, frequency analysis identifies and selectively filters out low-frequency, persistent interference components, preventing these specific noises from misleading the steady-state quality trend over the long term. Simultaneously, by incorporating vehicle motion state information to correct the smoothing results, the generation process of the steady-state quality trend can adapt to the vehicle's dynamic driving conditions, effectively avoiding errors caused by simple smoothing in complex and changing environments. Therefore, the obtained steady-state quality trend more accurately reflects the true quality changes of both visual and non-visual information, providing a more reliable basis for subsequent calculation of calibration parameter update weights, thereby improving the overall performance and adaptability of the visual sensor calibration algorithm.

[0098] In some preferred embodiments, a specific example is given below. Suppose a vehicle is driving at night, passing through an area with dense streetlights and slight flickering light. In this situation, the visual information quality score may exhibit periodic low-frequency fluctuations. These fluctuations are not caused by a degradation in the performance of the visual sensor itself, but rather by the characteristics of the ambient light. If only a simple moving average or exponential smoothing is used, these low-frequency fluctuations may be incorrectly smoothed out or misinterpreted as a genuine degradation in sensor quality, thus affecting subsequent weight calculations.

[0099] According to the scheme of this application, frequency analysis is first performed on the instantaneous quality score. Through Fourier transform, low-frequency, persistent interference components corresponding to the streetlight flicker frequency can be identified. For example, if a streetlight flickers at a frequency of 50Hz or 60Hz, and this flicker manifests as low-frequency brightness or contrast fluctuations in the image acquired by the visual sensor, frequency analysis will be able to detect this feature.

[0100] When such low-frequency interference is detected, the system dynamically adjusts the parameters of the smoothing filter based on its frequency characteristics. For example, a narrowband bandstop filter can be configured with a center frequency that matches the detected interference frequency to accurately suppress the interference component while preserving other useful information to the maximum extent possible.

[0101] Furthermore, assuming the vehicle maintains a stable speed while traversing this road segment, but the visual quality score fluctuates significantly due to streetlight flickering, the system, after smoothing, incorporates vehicle motion state information (e.g., stable speed and acceleration) to correct the smoothing result. If the vehicle's motion is stable, the reliability of the smoothed quality trend can be further confirmed, and the smoothing effect may be enhanced to some extent, ensuring that the final stable quality trend accurately reflects the visual information quality under stable driving conditions, after eliminating environmental interference.

[0102] In this way, even in complex scenarios with specific environmental disturbances or changes in vehicle motion, the proposed solution can generate more accurate and robust steady-state quality trends, thereby providing more reliable input for the visual sensor calibration algorithm and ensuring the accuracy and stability of the calibration.

[0103] In some embodiments of this application, when the environment switching confidence level remains above a preset threshold and non-visual information remains stable, the system determines that it has fully entered the standard environment type and triggers rapid calibration correction. However, relying solely on the environment switching confidence level and the continuous stability of non-visual information to determine whether a standard environment type has been fully entered may have limitations such as insufficient precision or robustness, especially in complex transition regions, which may lead to inaccurate timing of rapid calibration correction triggering. Therefore, this application further proposes a more refined and reliable triggering mechanism to ensure that rapid calibration correction can be accurately triggered when the environment is truly stable and meets standard conditions. When the environment type is switched from the specific misleading environment type to the standard environment type, a fast calibration correction is triggered, including: Continuously acquire visual and non-visual information; The degree of matching of standard features in the visual perception information is evaluated to generate a visual feature matching score; The stationarity of the non-visual information is evaluated, and a stationarity score for the non-visual information is generated. The visual feature matching score and the non-visual information stability score are fused to generate a comprehensive environmental reliability score, and a rapid calibration correction is triggered based on the comprehensive environmental reliability score.

[0104] Specifically, continuously acquiring visual and non-visual perception information means that the vehicle's perception system continuously receives data streams from onboard visual sensors (such as cameras) and non-visual sensors (such as inertial measurement units, high-precision positioning systems, radar, lidar, etc.). This continuous data acquisition aims to provide a real-time and comprehensive information foundation for subsequent environmental assessments.

[0105] The evaluation of the matching degree of standard features in the visual perception information, generating a visual feature matching score, can be understood as the system comparing the current visual perception information with predefined standard environmental features. Standard features typically refer to visual elements that are stable and clearly identifiable in typical standard environments, such as clear road markings, traffic signs, lane lines, and building edges. The evaluation of the matching degree can be achieved through various computer vision algorithms. For example, feature point detection and matching algorithms (such as SIFT, SURF, ORB, etc.) can be used to identify and match key feature points in the image, or deep learning models can be used for semantic segmentation or object detection to quantify the similarity between the current visual scene and the standard environmental feature model. This generates a visual feature matching score between 0 and 1, with a higher score indicating a more standard-compliant visual environment.

[0106] In practical applications, evaluating the stationarity of non-visual information and generating a non-visual information stationarity score involves performing stability analysis on the data output from vehicle non-visual sensors (such as GNSS and IMU). Non-visual information typically includes the vehicle's position, speed, and attitude. Stationarity assessment can be performed by calculating statistics (such as variance, standard deviation, and rate of change) of the non-visual data within a certain time window. For example, when the positioning accuracy of the GNSS signal fluctuates little, or when the acceleration and angular velocity readings of the IMU remain stable over a period of time, it indicates that the quality of the non-visual information is high and the environment is stable; in this case, the non-visual information stationarity score will be correspondingly higher. This score can also be quantified as a value between 0 and 1.

[0107] Furthermore, fusing the visual feature matching score and the non-visual information stationary score to generate a comprehensive environmental reliability score involves comprehensively considering the evaluation results of the two different modalities. The fusion method can employ weighted summation, product, or other non-linear combinations. For example, different weights can be assigned to the visual feature matching score and the non-visual information stationary score based on the reliability of different sensors in a specific environment. Thus, the comprehensive environmental reliability score, as a unified indicator, can more comprehensively and accurately reflect the reliability of the current environment as a standard environment.

[0108] Finally, a rapid calibration correction is triggered based on the comprehensive environmental reliability score. When the comprehensive environmental reliability score reaches or exceeds a preset threshold, the system considers the current environment to have highly reliable standard environmental characteristics, at which point the rapid calibration correction process of the vision sensor can be triggered safely and promptly.

[0109] This application's solution introduces and fuses visual feature matching scores and non-visual information stability scores to form a comprehensive environmental reliability score. This addresses the limitations of relying solely on environmental switching confidence and the continuous stability of non-visual information to determine whether a vehicle has fully entered a standard environment type. Through this technical solution, this application significantly improves the accuracy and reliability of rapid calibration correction triggering for vehicle vision sensors. By comprehensively evaluating the matching degree of visual features and the stability of non-visual information, the system can more accurately determine whether the vehicle has completely left a specific misleading environment and entered a stable standard environment. This effectively avoids problems such as incorrect calibration timing or poor calibration results due to inaccurate environmental judgment, thereby improving the adaptability and robustness of vision sensors during complex environment switching processes and ensuring the continuous high-precision operation of the vehicle perception system.

[0110] In some preferred embodiments, this application is implemented as follows. Assume an autonomous vehicle exits a dimly lit, feature-sparsely characterized underground parking lot (a specific misleading environment type) and enters a brightly lit, clearly marked city street (a standard environment type). As the vehicle exits the parking lot, the system continuously acquires visual and non-visual information from the onboard camera and GNSS / IMU unit. The visual perception module analyzes the images captured by the camera in real time, identifying and evaluating the matching degree of standard features such as road markings, traffic signs, and building outlines. For example, when the vehicle has completely exited the parking lot, and the road markings are clearly visible and highly consistent with a preset standard pattern, the visual feature matching score will rapidly increase, for example, reaching 0.9. Simultaneously, the non-visual information module monitors the purity of the GNSS signal and the stability of the IMU data. When the vehicle is driving in an open area, and the GNSS signal stabilizes and the IMU data fluctuations are small, the non-visual information stability score will also increase accordingly, for example, reaching 0.95. The system then weights and fuses these two scores; for example, the visual feature matching score is multiplied by 0.6, and the non-visual information stability score is multiplied by 0.4, to obtain the comprehensive environmental reliability score. In this example, the comprehensive environmental reliability score = 0.9 * 0.6 + 0.95 * 0.4 = 0.54 + 0.38 = 0.92. When this comprehensive environmental reliability score remains above a preset threshold (e.g., 0.85) for a period of time, the system determines that the vehicle has fully entered a standard environmental type and immediately triggers a rapid calibration correction to ensure that the visual sensor parameters can quickly adapt to the new environmental conditions and restore optimal perception performance.

[0111] In some of the embodiments described above in this application, a scheme is proposed that integrates visual feature matching scores and non-visual information stationary scores to generate a comprehensive environmental reliability score, and triggers rapid calibration correction based on this score. However, in its implementation, triggering based solely on the instantaneous comprehensive environmental reliability score may be sensitive to short-term environmental fluctuations, leading to unstable triggering or premature activation of the rapid calibration correction, thereby affecting the accuracy and reliability of the calibration.

[0112] In this regard, this application further proposes that the steps for triggering rapid calibration correction based on the comprehensive environmental reliability score include: Monitor the comprehensive environmental reliability score and compare it with a preset first reliability threshold; When the comprehensive environmental reliability score is consistently higher than the first reliability threshold, and no mixed features are detected in the visual perception information, it is determined that the current environment has standard features; Start the standard feature continuous verification timer; During the operation of the standard feature persistence verification timer, the visual feature matching score and the non-visual information stability score are continuously monitored, and the comprehensive environmental reliability score is ensured to remain above a preset second reliability threshold. When the standard feature persistence verification timer reaches the preset duration and the comprehensive environmental reliability score is always higher than the second reliability threshold, it is determined that the standard environment type has been fully entered. Trigger a fast calibration correction.

[0113] Specifically, the comprehensive environmental reliability score refers to the system continuously acquiring and analyzing a comprehensive environmental reliability score that is a fusion of visual feature matching scores and non-visual information stationary scores. This score is compared in real time with a preset first reliability threshold to preliminarily determine whether the current environment is beginning to exhibit the characteristics of a standard environment. The first reliability threshold can be set according to the actual application scenario and the requirements for environmental reliability; for example, it can be set to 0.7 or 0.8.

[0114] When the overall environmental reliability score not only consistently exceeds the first reliability threshold, but also no mixed features are detected in the visual perception information, the system determines that the current environment initially possesses standard characteristics. Mixed features refer to the simultaneous presence of characteristics of both a specific misleading environment and a standard environment in the visual information during environment switching. For example, at a tunnel exit, there are both dark features from inside the tunnel and bright features from outside. The absence of detected mixed features ensures the purity of the visual information, further enhancing the accuracy of the standard environment judgment.

[0115] Based on this, the system will start a standard feature persistence verification timer. The purpose of this timer is to verify whether the standard features of the current environment are persistent, rather than transient fluctuations.

[0116] During the standard feature persistence verification timer operation, the system continuously monitors the visual feature matching score and the non-visual information stationarity score, ensuring that the overall environmental reliability score remains consistently above a preset second reliability threshold. The second reliability threshold can be the same as or higher than the first reliability threshold to provide a more stringent persistence verification standard. For example, if the first reliability threshold is 0.7, the second reliability threshold can be set to 0.75 or 0.8. Continuous monitoring ensures that environmental reliability remains at a high level throughout the timer's operation.

[0117] When the standard feature persistence verification timer reaches a preset duration, such as 5 or 10 seconds, and the overall environmental reliability score remains above the second reliability threshold throughout this period, the system can confidently conclude that it has fully entered the standard environment type. The preset duration can be adjusted based on vehicle speed, frequency of environmental changes, and requirements for calibration stability.

[0118] Finally, once the system confirms that it has fully entered the standard environment type, it will trigger a rapid calibration correction to ensure that the calibration process is carried out under stable and reliable environmental conditions.

[0119] This application's solution effectively addresses the instability issues that may arise from triggering rapid calibration correction solely based on instantaneous comprehensive environmental reliability scores by introducing multi-stage judgment logic and a continuous verification mechanism. Through this technical solution, the application significantly improves the stability and reliability of rapid calibration correction for vehicle vision sensors. Compared to solutions relying solely on instantaneous environmental reliability scores, this application, by introducing a continuous verification timer and multiple judgment conditions, effectively avoids unnecessary or inaccurate rapid calibration when the environment is unstable or exhibits transitional characteristics. This ensures that rapid calibration correction is only triggered when environmental conditions are truly stable and meet standard requirements, thereby improving the accuracy of calibration results, reducing safety risks caused by improper calibration, and enhancing the overall performance of the vehicle perception system.

[0120] In some preferred embodiments, a specific example is given below. Suppose an autonomous vehicle is exiting a long tunnel, the interior of which is a typical type of misleading environment. As the vehicle gradually exits the tunnel, the ambient light and visual characteristics begin to change.

[0121] First, the system continuously monitors the overall environmental reliability score. As the vehicle exits the tunnel, the light gradually becomes brighter, visual features (such as road markings and traffic signs) become clearer, and non-visual information (such as GPS signals) becomes more stable. At this point, the overall environmental reliability score gradually increases.

[0122] When the overall environmental reliability score continues to be higher than the preset first reliability threshold (e.g., 0.7), and the visual perception information no longer detects mixed features such as dim light and blur inside the tunnel, the system will initially determine that the current environment has standard characteristics.

[0123] Subsequently, the system immediately starts a standard feature persistence verification timer, for example, set to 5 seconds. During these 5 seconds, the system continuously monitors the visual feature matching score and the non-visual information stability score, ensuring that the overall environmental reliability score remains above a preset second reliability threshold (e.g., 0.75). This means that during these 5 seconds after the vehicle exits the tunnel, the environment must maintain stable and high-quality visual and non-visual information.

[0124] If the overall environmental reliability score remains above 0.75 within these 5 seconds, and there are no signs of environmental instability (such as suddenly entering a dense fog or encountering strong backlight interference), then when the timer reaches 5 seconds, the system will determine that the vehicle has fully entered the standard environmental type.

[0125] Ultimately, the system triggers a rapid calibration correction, quickly adjusting the vision sensor parameters using the current stable and high-quality environmental information, and then resuming the normal calibration mode. This mechanism effectively avoids calibration when the vehicle has just exited the tunnel and the environment may still have brief fluctuations, thus ensuring the accuracy and stability of the calibration.

[0126] In some embodiments described above, this application proposes continuously monitoring the visual feature matching score and the stationary score of non-visual information during the operation of the standard feature persistence verification timer, ensuring that the overall environmental reliability score remains above a preset second reliability threshold. However, in practical applications, visual perception information and non-visual information may be affected by instantaneous environmental changes or sensor noise, causing temporary fluctuations in the visual feature matching score and the stationary score of non-visual information. If these instantaneous fluctuations cause the overall environmental reliability score to temporarily fall below the preset threshold, it may erroneously interrupt the standard feature persistence verification timer, thereby affecting the accurate judgment of the standard environment type and the timely triggering of rapid calibration correction. To address this, this application further proposes a more robust monitoring mechanism, which effectively eliminates the impact of instantaneous fluctuations by performing time series analysis and short-term smoothing on the visual feature matching score and the stationary score of non-visual information, ensuring that the evaluation of the overall environmental reliability score is more stable and accurate.

[0127] The above-mentioned continuous monitoring of visual feature matching scores and non-visual information stationary scores during the standard feature persistence verification timer operation, and ensuring that the comprehensive environmental reliability score remains above a preset second reliability threshold, includes: Continuously acquire the visual feature matching score and the non-visual information stationary score; Time series analysis is performed on the visual feature matching score and the stationary score of non-visual information to identify whether there are instantaneous fluctuations. When the instantaneous fluctuation is detected, the visual feature matching score and the non-visual information stability score are subjected to short-term smoothing to eliminate the influence of the instantaneous fluctuation. The comprehensive environmental reliability score is recalculated based on the smoothed visual feature matching score and the non-visual information stationarity score. Compare the recalculated comprehensive environmental reliability score with the preset second reliability threshold; When the recalculated comprehensive environmental reliability score continues to be higher than the second reliability threshold, the standard feature persistence verification timer continues to run.

[0128] Specifically, continuously acquiring visual feature matching scores and non-visual information stability scores refers to the system continuously receiving real-time data from the visual perception module and non-visual sensor module at a preset sampling frequency during the standard feature persistence verification timer operation, and calculating the corresponding visual feature matching scores and non-visual information stability scores. These scores reflect the degree of matching between the visual features of the current environment and the standard features, as well as the stability of non-visual information.

[0129] The process of performing time-series analysis on visual feature matching scores and stationary scores of non-visual information to identify the presence of transient fluctuations can be understood as using statistical or signal processing methods, such as moving averages, exponential smoothing, and Kalman filtering, to analyze continuously acquired score sequences. The aim is to distinguish between persistent trends caused by changes in the real environment and transient, non-persistent fluctuations caused by noise or brief disturbances. Transient fluctuations typically manifest as a sharp rise or fall in scores over a short period, followed by a rapid return to normal levels.

[0130] In practical applications, when transient fluctuations are detected, a short-term smoothing process is applied to the visual feature matching score and the stable score of non-visual information to eliminate the impact of the transient fluctuations. For example, a moving average filter, median filter, or low-pass filter can be used to process the affected scores. The window size or filtering parameters of the short-term smoothing process should be adjusted according to the typical duration of the transient fluctuation to preserve the trend of real environmental changes as much as possible while eliminating noise. The goal is to provide a more stable and representative score, avoiding misjudgments of the environmental state due to transient outliers.

[0131] Furthermore, the overall environmental reliability score is recalculated based on the smoothed visual feature matching score and the non-visual information stationary score. This recalculation process is the same as the original overall environmental reliability score calculation method, but the input data has been smoothed, thus making the new score more reflective of the true reliability of the environment.

[0132] Subsequently, the recalculated overall environmental reliability score is compared with a preset second reliability threshold. This threshold is used to define whether the environment is stable and reliable enough to trigger rapid calibration correction.

[0133] When the recalculated overall environmental reliability score remains consistently above the second reliability threshold, the standard feature persistence verification timer continues to run. This means that even if the original score fluctuates momentarily, the processed score still meets the requirements of the standard environment, thus ensuring that the timer is not accidentally interrupted and the verification process can continue.

[0134] This application's solution effectively addresses the issue of transient fluctuations affecting visual feature matching scores and stationary non-visual information scores during standard feature persistence verification by introducing time series analysis and short-term smoothing. Specifically, after continuously acquiring visual feature matching scores and stationary non-visual information scores, the system performs time series analysis to identify and distinguish between transient fluctuations caused by brief disturbances and persistent trends caused by changes in the real environment. Once a transient fluctuation is identified, the affected scores are immediately smoothed using methods such as moving averages or low-pass filtering to effectively filter out these non-persistent noise components, allowing the scores to more accurately reflect the true state of the current environment. Subsequently, the comprehensive environmental reliability score is recalculated based on these smoothed scores to ensure its stability. By comparing the recalculated comprehensive environmental reliability score with a preset second reliability threshold, and only maintaining the standard feature persistence verification timer running if the smoothed score remains consistently above the threshold, this application's solution avoids misjudgments caused by transient fluctuations, thereby ensuring that the timer accurately reflects the duration of the vehicle in the standard environment.

[0135] Through the above technical solution, this application can significantly improve the robustness and accuracy of the vehicle vision sensor calibration method in the transitional region of environmental switching. Specifically, by performing time series analysis and short-term smoothing on the visual feature matching score and the stationary score of non-visual information, the system can effectively filter out the influence of transient noise and brief interference on the comprehensive environmental reliability score, avoiding erroneous interruption of the standard feature continuous verification timer due to occasional fluctuations. This makes the judgment of the standard environment type more stable and reliable, reduces the occurrence of misjudgments, and ensures that rapid calibration correction can be triggered in a timely and accurate manner when truly entering a stable standard environment. Thus, it not only improves the efficiency of the calibration process but also enhances the reliability of the vehicle's perception system in complex dynamic environments.

[0136] In some preferred embodiments, a specific example is given below. Suppose a vehicle exits a tunnel and enters an open city road. After the standard feature persistence verification timer starts, the system continuously acquires visual feature matching scores and non-visual information stationary scores. At some point, due to brief direct sunlight or momentary occlusion by a roadside vehicle, the visual feature matching score may experience a brief drop, while the non-visual information stationary score may also fluctuate slightly due to minor bumps.

[0137] At this point, the system immediately performs time-series analysis on these scores. For example, a 5-second moving average window is used to process the visual feature matching scores and the non-visual information stationary scores. If the original score suddenly drops at a certain sampling point but recovers rapidly in the next few sampling points, the score after moving average processing will not drop significantly, but will remain at a relatively stable level.

[0138] Specifically, suppose that at second T, the original visual feature matching score drops from 0.9 to 0.6, but recovers to 0.85 by second T+1. If a 5-second moving average is used, the moving average score at second T will include data from second T-4 to second T, and its decline will be smoothed out. At second T+1, the new moving average score will include data from second T-3 to second T+1, and its value will quickly rebound.

[0139] Based on these smoothed visual feature matching scores and non-visual information stationary scores, the system recalculates the overall environmental reliability score. For example, if both the smoothed visual feature matching score and the non-visual information stationary score remain consistently above 0.8, the recalculated overall environmental reliability score will also remain consistently above a preset second reliability threshold (e.g., 0.75). In this case, even if the original score fluctuates momentarily, the standard feature persistence verification timer will continue to run without interruption. Only when the smoothed overall environmental reliability score remains consistently below the threshold will the system be judged as not having fully entered the standard environment, thus avoiding misjudgments caused by transient interference and ensuring the stability and accuracy of the calibration process.

[0140] refer to Figure 4 , Figure 4 This is a schematic diagram of a vehicle vision sensor calibration system provided in an embodiment of the present invention, comprising: The input terminal is used to acquire vehicle location information, preset environmental information, and visual perception information. The triggering end is used to determine the current environment type based on the location information, the preset environment information, and the visual perception information; when the environment type is a specific misleading environment type, it monitors and records the deviation information between the visual perception result and the non-visual information; when the environment type switches from the specific misleading environment type to the standard environment type, it triggers a fast calibration correction. The correction end is used to compare the current parameters of the vision sensor with the ideal parameters calculated based on standard visual information and the non-visual information. Based on the comparison results and the deviation information, the vision sensor parameters are quickly corrected and the normal calibration mode is restored.

[0141] This system employs a modular design, separating information acquisition, environmental assessment and calibration triggering, and parameter correction and mode recovery functions into input, triggering, and correction ends, respectively. This structured approach effectively addresses the challenges of calibrating visual sensors in intelligent vehicles under complex dynamic environments, particularly resolving the issues of error accumulation in misleading environments and slow or divergent system response during environmental transitions in traditional methods. Through the collaborative work of its functional modules, this system achieves intelligent, rapid, and robust calibration of visual sensor parameters, thereby significantly improving the perception accuracy and driving safety of intelligent vehicles.

[0142] In some embodiments of this application, the specific steps and principles of the vehicle vision sensor calibration method have already been described in the above embodiments, and will not be repeated here. It should be emphasized that the vehicle vision sensor calibration system proposed in this application, through its specific structured components, transforms these method steps into executable functional modules.

[0143] Specifically, the input end can be understood as the system's data acquisition module, whose function is to continuously acquire various environmental and status information required for vehicle operation. For example, the input end may include one or more data interfaces for receiving position information from the Global Navigation Satellite System (GNSS) module, preset environmental information read from the vehicle's high-precision map database, and visual perception information acquired through the vehicle's camera array. As one implementation, the input end can be an integrated data bus or communication interface responsible for aggregating and initially processing these heterogeneous data streams. In practical applications, the input end can also be configured with a data preprocessing unit to perform format conversion, time synchronization, or preliminary filtering of the raw data to ensure the efficiency and accuracy of subsequent processing.

[0144] The trigger can be understood as the system's intelligent decision-making and state management module. Its core function is to determine the vehicle's environment type in real time based on information provided by the input terminal and trigger corresponding calibration strategies according to environmental changes. For example, the trigger can be an embedded processor or a software module on an onboard computing platform. It runs an environment judgment algorithm to comprehensively analyze location information, preset environment information, and visual perception information. When a specific misleading environment type is determined, the trigger will activate a deviation information monitoring and recording mechanism; when it detects a switch from a specific misleading environment to a standard environment, the trigger will immediately issue a rapid calibration correction command. As one implementation method, the trigger can include a state machine that transitions according to different environmental conditions and internal system states, and controls the initiation and switching of the calibration process.

[0145] The correction unit can be understood as the core calibration execution module of the system. Its function is to quickly and accurately adjust the parameters of the vision sensor according to the instructions issued by the trigger unit. For example, the correction unit can be a dedicated digital signal processor (DSP) or graphics processing unit (GPU) to execute complex calibration algorithms. It receives standard visual and non-visual information from the input unit and compares it with the current parameters of the vision sensor, while also incorporating the deviation information recorded by the trigger unit. Based on these inputs, the correction unit calculates the optimal parameter correction amount and applies it to the vision sensor. As one implementation, the correction unit can include a parameter optimizer, which gradually adjusts the sensor parameters through iterative or closed-loop control until the ideal state is reached. After completing the rapid correction, the correction unit is also responsible for restoring the system to the normal calibration mode to ensure the continuity and stability of the calibration.

[0146] The vehicle vision sensor calibration system proposed in this application provides a structured and efficient solution to the problems of low calibration accuracy, poor robustness, and potential divergence or slow response of the calibration system when the environment changes in the prior art.

[0147] Traditional calibration systems often employ a single calibration strategy, making it difficult to adapt to rapid environmental changes, especially in misleading environments such as highway reconstruction and expansion sites, where systematic parameter deviations can easily accumulate. In contrast, the calibration system proposed in this application introduces functional modules such as an input terminal, a trigger terminal, and a correction terminal, enabling intelligent judgment of environmental types and differentiated calibration strategies. The input terminal comprehensively acquires multi-source information, providing a reliable basis for subsequent decisions. The trigger terminal, acting as an intelligent decision-making center, can identify the switch between specific misleading environments and standard environments in real time and initiate the corresponding calibration mode based on environmental characteristics, avoiding blind calibration in unreliable environments. The correction terminal, by combining historical deviation information, performs rapid and accurate parameter correction during environmental changes, effectively eliminating systematic deviations accumulated in misleading environments and avoiding calibration divergence or response lag problems that may occur in traditional systems during drastic environmental changes.

[0148] Through this systematic design, the calibration system of this application not only improves the calibration accuracy and robustness of vision sensors in complex environments, but also significantly shortens the calibration recovery time when switching from a misleading environment to a standard environment, thereby ensuring the continuous and safe operation of intelligent vehicle assisted driving functions. This modular and intelligent calibration system demonstrates significant advancements in adaptability, accuracy, and response speed compared to existing technologies.

[0149] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for calibrating a vehicle vision sensor, characterized in that, include: Acquire vehicle location information, preset environmental information, and visual perception information; The current environment type is determined based on the location information, the preset environment information, and the visual perception information; When the environment type is a specific misleading environment type, the deviation information between visual perception results and non-visual information is monitored and recorded; When the environment type is switched from the specific misleading environment type to the standard environment type, a fast calibration correction is triggered; The current parameters of the vision sensor are compared with the ideal parameters calculated based on standard visual information and the non-visual information. Based on the comparison results and the deviation information, the vision sensor parameters are quickly corrected and the normal calibration mode is restored.

2. The vehicle vision sensor calibration method according to claim 1, characterized in that, The comparison involves comparing the current parameters of the visual sensor with ideal parameters calculated based on standard visual information and the non-visual information. Based on the comparison results and the deviation information, the visual sensor parameters are then rapidly corrected, including: The contrast, edge integrity, continuity, and degree of matching with standard patterns of visual features are obtained to generate visual feature quality scores. Monitor the purity of the output signals from the inertial measurement unit and the high-precision positioning system, and generate a purity score for the non-visual sensors; The visual feature quality score and the non-visual sensor purity score are combined to generate a comprehensive information quality score. Based on the comprehensive information quality score, the calibration correction factor is calculated, and the update of the calibration parameters in the vision sensor calibration algorithm is adjusted accordingly. Based on the deviation information, the total deviation between the current parameters of the visual sensor and the ideal parameters calculated based on the standard visual information and the non-visual information is compared with the deviation information. Based on the comparison results, systematic deviations are eliminated or conservative corrections are made.

3. The vehicle vision sensor calibration method according to claim 1, characterized in that, When the environment type is a specific misleading environment type, the deviation information between visual perception results and non-visual information is monitored and recorded, including: The non-visual information is continuously acquired, and the purity of the non-visual information is evaluated to generate a non-visual information purity score. When the purity score of the non-visual information is lower than a preset threshold, non-visual information drift detection is initiated. Analyze the changing trend of the non-visual information within a preset time period to determine whether the non-visual information has drifted. When a drift in the non-visual information is detected, the visual information-assisted verification mechanism is activated. After the non-visual information is corrected, the deviation between the visual perception result and the non-visual information is monitored and recorded.

4. The vehicle vision sensor calibration method according to claim 1, characterized in that, When the environment type is switched from the specific misleading environment type to the standard environment type, a fast calibration correction is triggered, including: Continuously acquire visual and non-visual information; Analyze whether there are mixed features in the visual perception information, and evaluate the stability of the non-visual information in the switching region; The clarity, completeness, and continuity of standard features in the visual perception information, as well as the stability and accuracy of the non-visual information, are obtained to generate an environment switching confidence score. When the confidence level of the environment switch reaches a preset threshold, it is determined that the current area is in the transition zone, and the progressive calibration correction mode is started in the transition zone. Based on the confidence level of the environment switch, the weights of the visual information and the non-visual information in the calibration parameter update are dynamically adjusted. When the confidence level of the environment switching is consistently higher than the preset threshold and the non-visual information remains stable, it is determined that the system has fully entered the standard environment type, and a rapid calibration correction is triggered.

5. The vehicle vision sensor calibration method according to claim 4, characterized in that, The step of dynamically adjusting the weights of visual and non-visual information in the calibration parameter update based on the environmental switching confidence level includes: Continuously acquire the visual information and the non-visual information; The visual information and the non-visual information are evaluated for quality, and corresponding instantaneous quality scores are generated. The instantaneous quality score is smoothed over time to obtain a stable quality trend; Based on the steady-state quality trend and the confidence level of the environment switching, calculate the weights of the visual information and the non-visual information in the calibration parameter update; The update of calibration parameters in the visual sensor calibration algorithm is adjusted according to the weights.

6. The vehicle vision sensor calibration method according to claim 5, characterized in that, The step of performing time-series smoothing on the instantaneous quality score to obtain a stable quality trend includes: Frequency analysis is performed on the instantaneous quality score to identify whether there are low-frequency, persistent interference components in the instantaneous quality score; When the low-frequency, persistent interference component is identified, the parameters of the smoothing filter are adjusted according to the frequency characteristics of the interference component to smooth the instantaneous quality score. By combining vehicle motion status information, the smoothing results are corrected to obtain a stable quality trend.

7. The vehicle vision sensor calibration method according to claim 4, characterized in that, When the confidence level of the environment switch remains higher than the preset threshold and the non-visual information remains stable, it is determined that the system has fully entered the standard environment type, and a rapid calibration correction is triggered, including: Continuously acquire visual and non-visual information; The degree of matching of standard features in the visual perception information is evaluated to generate a visual feature matching score; The stationarity of the non-visual information is evaluated, and a stationarity score for the non-visual information is generated. The visual feature matching score and the non-visual information stability score are fused to generate a comprehensive environmental reliability score, and a rapid calibration correction is triggered based on the comprehensive environmental reliability score.

8. The vehicle vision sensor calibration method according to claim 7, characterized in that, The step of triggering rapid calibration correction based on the comprehensive environmental reliability score includes: Monitor the comprehensive environmental reliability score and compare it with a preset first reliability threshold; When the comprehensive environmental reliability score is consistently higher than the first reliability threshold, and no mixed features are detected in the visual perception information, it is determined that the current environment has standard features; Start the standard feature continuous verification timer; During the operation of the standard feature persistence verification timer, the visual feature matching score and the non-visual information stability score are continuously monitored, and the comprehensive environmental reliability score is ensured to remain above a preset second reliability threshold. When the standard feature persistence verification timer reaches the preset duration and the comprehensive environmental reliability score is always higher than the second reliability threshold, it is determined that the standard environment type has been fully entered. Trigger a fast calibration correction.

9. A vehicle vision sensor calibration method according to claim 8, characterized in that, During the operation of the standard feature persistence verification timer, the visual feature matching score and the non-visual information stability score are continuously monitored, and the overall environmental reliability score is ensured to remain above a preset second reliability threshold, including: Continuously acquire the visual feature matching score and the non-visual information stationary score; Time series analysis is performed on the visual feature matching score and the stationary score of non-visual information to identify whether there are instantaneous fluctuations. When the instantaneous fluctuation is detected, the visual feature matching score and the non-visual information stability score are subjected to short-term smoothing to eliminate the influence of the instantaneous fluctuation. The comprehensive environmental reliability score is recalculated based on the smoothed visual feature matching score and the non-visual information stationarity score. Compare the recalculated comprehensive environmental reliability score with the preset second reliability threshold; When the recalculated comprehensive environmental reliability score continues to be higher than the second reliability threshold, the standard feature persistence verification timer continues to run.

10. A vehicle vision sensor calibration system, characterized in that, include: The input terminal is used to acquire vehicle location information, preset environmental information, and visual perception information. The triggering end is used to determine the current environment type based on the location information, the preset environment information, and the visual perception information; When the environment type is a specific misleading environment type, the deviation information between visual perception results and non-visual information is monitored and recorded; when the environment type is switched from the specific misleading environment type to the standard environment type, a rapid calibration correction is triggered. The correction end is used to compare the current parameters of the vision sensor with the ideal parameters calculated based on standard visual information and the non-visual information. Based on the comparison results and the deviation information, the vision sensor parameters are quickly corrected and the normal calibration mode is restored.

Images