Image anti-shake processing method and related product
By identifying and filtering out the vehicle's human operation jitter and only compensating for pothole jitter, the problem of frequent jumps in the AR-HUD display screen is solved, and the driver's visual perception and safety are improved.
Patent Information
- Application Number
- CN202411388172.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2024-11-15
AI Technical Summary
When the vehicle uses the enhanced display head-up display (AR-HUD) technology, the display screen jumps frequently, affecting the driver's visual perception and safety.
By obtaining the vehicle's target pitch angle change and acceleration, identifying and filtering out human operation jitter, only reverse compensation is performed for pothole jitter, and AR-HUD display image is adjusted.
It effectively avoids the impact of artificial operation jitter on the AR-HUD display screen, improves the stability and visualization of the display screen, and improves the driver's visual perception and safety.
Smart Images

Figure CN118972699A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and in particular to an image anti-shake processing method and related products. Background Art
[0002] Augmented Reality Head-Up Display (AR-HUD) technology is a technology that overlays virtual information into the driver's field of view to improve the driver's visual perception and safety. However, when a vehicle uses AR-HUD technology for AR-HUD display, the AR-HUD display screen often jumps frequently, causing the driver's attention to be distracted during driving, affecting the driver's driving safety. Summary of the invention
[0003] The embodiments of the present application provide an image stabilization processing method and related products, which are used to avoid frequent jumping of the AR-HUD display screen, improve the display stability and visualization effect of the AR-HUD, and thus improve the driver's visual perception and safety.
[0004] In order to achieve the above purpose, the embodiment of the present application adopts the following technical solution:
[0005] In a first aspect, an embodiment of the present application provides an image stabilization processing method, the method comprising:
[0006] Acquire a target pitch angle change amount and a vehicle acceleration of the vehicle at a first moment, wherein the target pitch angle change amount indicates a change amount of the pitch angle at the first moment relative to the pitch angle at a second moment;
[0007] If the vehicle acceleration meets a preset condition, determining that the vehicle vibration includes human-operated vibration; filtering the first pitch angle variation from the target pitch angle variation to obtain a second pitch angle variation; the first pitch angle variation indicates a variation of the pitch angle caused by human-operated vibration at the first moment relative to the pitch angle caused by human-operated vibration at the second moment;
[0008] If the second pitch angle change is greater than the first change threshold, the second pitch angle change is reversely compensated to obtain a corrected pitch angle change of the vehicle at the first moment; and the AR-HUD display image is determined using the corrected pitch angle change.
[0009] Optionally, the preset condition includes a first preset condition, a second preset condition and a third preset condition, the first preset condition indicating that the vehicle acceleration belongs to a first acceleration interval corresponding to sudden deceleration, the second preset condition indicating that the vehicle acceleration belongs to a second acceleration interval corresponding to sudden acceleration, and the third preset condition indicating that the vehicle acceleration belongs to a third acceleration interval corresponding to starting from a stationary state;
[0010] If the vehicle acceleration satisfies a preset condition, determining that the vehicle vibration includes the human-operated vibration includes: if the vehicle acceleration satisfies the first preset condition, the second preset condition or the third preset condition, determining that the vehicle vibration includes the human-operated vibration.
[0011] Optionally, the method further comprises:
[0012] Based on the first speed interval, performing a pre-operation of sudden deceleration, a pre-operation of sudden acceleration and a pre-operation of starting at rest on a smooth road section, and acquiring the first acceleration interval, the second acceleration interval and the third acceleration interval; the smooth road section indicates a road section where the fluctuation amplitude value is less than the fluctuation amplitude threshold;
[0013] Among them, the first speed interval is the interval from the first speed value to the second speed value, and the first speed value is less than the second speed value; the rapid acceleration pre-operation indicates an operation of accelerating the speed value of the vehicle from the first speed value to the second speed value within a preset time; the rapid deceleration pre-operation indicates an operation of decelerating the speed value of the vehicle from the second speed value to the first speed value within a preset time; the stationary starting pre-operation indicates that the speed value of the vehicle is accelerated from a stationary state to a stable speed.
[0014] Optionally, determining a first sub-pitch angle change amount corresponding to the first acceleration interval, a second sub-pitch angle change amount corresponding to the second acceleration interval, and a third sub-pitch angle change amount corresponding to the third acceleration interval;
[0015] The filtering out the first pitch angle variation from the target pitch angle variation to obtain the second pitch angle variation includes:
[0016] If the vehicle acceleration satisfies the first acceleration interval, the first sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change; if the vehicle acceleration satisfies the second acceleration interval, the second sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change; if the vehicle acceleration satisfies the third acceleration interval, the third sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change.
[0017] Optionally, the method further comprises:
[0018] Acquire a yaw angle change and a roll angle change of the vehicle at the first moment; the yaw angle change indicates a change of the yaw angle at the first moment relative to the yaw angle at the second moment, and the roll angle change indicates a change of the roll angle at the first moment relative to the roll angle at the second moment;
[0019] The method further comprises:
[0020] If the vehicle acceleration does not meet the preset condition, it is determined whether the vibration includes the human-operated vibration according to the yaw angle change and the roll angle change.
[0021] Optionally, if the vehicle acceleration satisfies a preset condition, determining that the vehicle vibration includes human-operated vibration includes:
[0022] If the driving section of the vehicle at the first moment is a pothole section or a smooth section, and if the acceleration of the vehicle meets a preset condition, determining that the vibration of the vehicle includes human-operated vibration;
[0023] If the target pitch angle change is less than a second change threshold, and the vehicle's gravity acceleration change is less than the gravity acceleration change threshold, it is determined that the driving section of the vehicle at the first moment is a pit entry section; reversely compensate the target pitch angle change to obtain the corrected pitch angle change.
[0024] Optionally, the method of using the modified pitch angle variation to determine an augmented reality head-up display AR-HUD display image includes:
[0025] If the modified pitch angle variation is greater than a third variation threshold, processing the modified pitch angle variation based on a smoothing algorithm and superimposing reverse compensation; and determining the AR-HUD display image using the processed modified pitch angle variation;
[0026] or,
[0027] Preprocessing the corrected pitch angle variation to obtain a corrected pitch angle variation sequence;
[0028] Filtering a plurality of interpolation points from the modified pitch angle variation sequence;
[0029] Based on a preset interpolation method, determining data values between the plurality of interpolation points to obtain a smoothed sequence of corrected pitch angle variation;
[0030] The AR-HUD display image is determined according to the smoothed modified pitch angle variation sequence.
[0031] Optionally, if the vehicle acceleration satisfies a preset condition, before determining that the vehicle vibration includes human-operated vibration, the method further includes:
[0032] filtering a third pitch angle variation caused by road noise jitter from the target pitch angle variation to obtain an initial corrected pitch angle variation;
[0033] The filtering out the first pitch angle change amount from the target pitch angle change amount to obtain the second pitch angle change amount comprises: filtering out the first pitch angle change amount from the initial corrected pitch angle change amount to obtain the second pitch angle change amount;
[0034] and / or,
[0035] Based on the second speed range, using the vehicle to test multiple road conditions that produce pothole jitters, and collecting jitter data for each of the multiple road conditions; the jitter data includes jitter frequency and pitch angle jitter amplitude;
[0036] Based on the jitter data of each road condition, the multiple road conditions are classified into a large-amplitude jitter road condition, a medium-amplitude jitter road condition and a small-amplitude jitter road condition;
[0037] Among them, the large-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is greater than the first amplitude threshold, and the shaking frequency is less than the first frequency threshold; the medium-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is greater than the second amplitude threshold, and less than or equal to the first amplitude threshold, and the shaking frequency is greater than or equal to the first frequency threshold, and less than the second frequency threshold; the small-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is less than or equal to the second amplitude threshold, and the shaking frequency is greater than or equal to the second frequency threshold;
[0038] The step of obtaining a target pitch angle change amount and a vehicle acceleration of the vehicle at a first moment includes:
[0039] If the road condition corresponding to the driving section of the vehicle at the first moment belongs to the target road condition, the target pitch angle change and vehicle acceleration of the vehicle at the first moment are obtained; the target road condition is at least one of the large-amplitude shaking road condition, medium-amplitude shaking road condition and small-amplitude shaking road condition.
[0040] In a second aspect, an embodiment of the present application provides an image stabilization processing device, the device comprising:
[0041] an acquisition unit, configured to acquire a target pitch angle change amount of the vehicle at a first moment and a vehicle acceleration, wherein the target pitch angle change amount indicates a change amount of the pitch angle at the first moment relative to the pitch angle at a second moment;
[0042] A determination unit, configured to determine that the jitter of the vehicle includes human-operated jitter if the vehicle acceleration satisfies a preset condition; filter out a first pitch angle change from the target pitch angle change to obtain a second pitch angle change; the first pitch angle change indicates a change in the pitch angle caused by human-operated jitter at the first moment relative to the pitch angle caused by human-operated jitter at the second moment;
[0043] The compensation display unit is used to reversely compensate the second pitch angle change if the second pitch angle change is greater than the first change threshold, so as to obtain the corrected pitch angle change of the vehicle at the first moment; and use the corrected pitch angle change to determine the AR-HUD display image.
[0044] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a memory and a processor;
[0045] The memory is coupled to the processor;
[0046] The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the method as described in any one of the first aspects.
[0047] Beneficial effects:
[0048] The embodiments of the present application provide an image anti-shake processing method and related products. When executing the method, by obtaining the target pitch angle change and vehicle acceleration of the vehicle at the first moment, the vehicle acceleration is used to determine whether the vehicle's jitter includes human-operated jitter. If there is human-operated jitter, the pitch angle change corresponding to the human-operated jitter (referred to as the first pitch angle change) is filtered out from the target pitch angle change of the vehicle. In other words, the embodiments of the present application identify human-operated jitter from the jitter of the vehicle, and filter out the human-operated jitter, retaining only the pothole jitter, and performing reverse compensation pitch angle processing on the pothole jitter, thereby avoiding the influence of human-operated jitter on the AR-HUD display screen, improving the phenomenon of frequent and violent jumping of the AR-HUD display screen, and helping to improve the driver's visual perception and safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 A flow chart of an image stabilization processing method provided in an embodiment of the present application;
[0050] Figure 2 A flowchart of another image stabilization processing method provided in an embodiment of the present application;
[0051] Figure 3 A schematic diagram of a vehicle traveling on a current road provided in an embodiment of the present application;
[0052] Figure 4 A flowchart of another image stabilization processing method provided in an embodiment of the present application;
[0053] Figure 5 An image stabilization processing device is provided for an embodiment of the present application. DETAILED DESCRIPTION
[0054] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0055] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0056] AR-HUD technology is a technology that combines augmented reality and head-up display. AR-HUD technology is used to superimpose virtual information on the real world, forming a virtual-real fusion effect, and displaying it in the driver's field of view, thereby improving the driver's visual perception and safety. For example, AR-HUD technology integrates the navigation, vehicle speed, warning and other information of the virtual machine with the actual road conditions to form a virtual-real fusion AR-HUD display screen, which is displayed in the driver's field of view to help the driver better control the vehicle, thereby improving the driver's driving safety. AR-HUD technology has a wide range of application scenarios and can be applied to various driving scenarios. For example, AR-HUD technology can be used in urban road navigation scenarios to clearly display road condition names, navigation arrows, etc., to assist drivers in making precise turns. For another example, AR-HUD technology can be used in long-distance driving scenarios on highways to display information such as vehicle speed and speed limit to improve driving safety.
[0057] The vehicle will vibrate during driving, which includes three types: road noise vibration, pothole vibration and human operation vibration. Among them, road noise vibration is caused by the slight vibration of the vehicle when driving due to the tiny bumps and depressions on the road surface. Road noise vibration is caused by the slight unevenness of the road surface, which cannot be completely avoided even on smooth roads. Pothole vibration refers to the strong vibration caused by the vehicle passing through pits, speed bumps or train tracks, which exist in the real world and are visible to the human eye. Pothole vibration is caused by the impact of uneven road surface on the wheels and body. Compared with road noise vibration, pothole vibration is more intense. Human operation vibration is caused by the sudden change of the vehicle pitch angle due to the driver's sudden acceleration, deceleration or sharp turn of the vehicle during driving. Road noise vibration, pothole vibration and human operation vibration will affect the display image of AR-HUD, so anti-shake processing is required.
[0058] At present, the anti-shake processing method mainly uses software algorithms to perform image stabilization processing, thereby reducing the impact of jitter on the AR-HUD display screen. Specifically, the real-time motion data of the vehicle measured by the inertial measurement unit (IMU) is analyzed to obtain the pitch angle of the vehicle. Based on the pitch angle of the vehicle, the display image of the AR-HUD is reversely compensated. Among them, the reverse compensation processing refers to determining the pitch angle change of the vehicle based on the pitch angle of the vehicle, using the pitch angle change of the vehicle to determine the compensation value (the compensation value is positively correlated with the pitch angle change of the vehicle), and using the compensation value to compensate for the pitch angle to offset the pitch angle change caused by vehicle jitter (ie, the pitch angle jitter amplitude). The processed AR-HUD display image is then output to the display screen of the AR-HUD for display, so as to reduce the impact of jitter on the display effect of the AR-HUD. However, when the vehicle is driving, the current anti-shake processing method often causes the AR-HUD display to shake. In addition, when the driver frequently performs human operations such as rapid acceleration, rapid deceleration, sharp turns, or starting at zero speed (when stationary), the AR-HUD display will frequently jump, affecting the driver's visual perception and safety.
[0059] The inventors have analyzed and found that the cause of the above problems is that although the vehicle will produce human operation jitter, road noise jitter and pothole jitter during driving, and this jitter will cause pitch angle changes, but compared with pothole jitter, the pitch angle changes caused by road noise jitter and human operation jitter are very small and can be basically ignored. However, if the vehicle is subject to human operation jitter and / or road noise jitter during driving, the current reverse compensation processing method will compensate for the pitch angle changes caused by road noise jitter and human operation jitter, and this compensation is often over-compensation, that is, the compensation amplitude and speed exceed the actual demand. Therefore, when the vehicle is subject to human operation jitter and / or road noise jitter during driving, the AR-HUD display screen often jumps frequently and violently.
[0060] In view of this, an embodiment of the present application provides an image anti-shake processing method, which obtains the target pitch angle change and vehicle acceleration of the vehicle at the first moment, and uses the vehicle acceleration to determine whether the vehicle's jitter includes human-operated jitter. If there is human-operated jitter, the pitch angle change corresponding to the human-operated jitter (referred to as the first pitch angle change) is filtered out from the target pitch angle change of the vehicle. In other words, the embodiment of the present application identifies human-operated jitter from the jitter of the vehicle, filters out the human-operated jitter, and only retains the pothole jitter, and performs reverse compensation pitch angle processing on the pothole jitter, thereby avoiding the influence of human-operated jitter on the AR-HUD display screen, improving the phenomenon of frequent and violent jumping of the AR-HUD display screen, and helping to improve the driver's visual perception and safety.
[0061] The following is a detailed description of the image stabilization processing method provided by the embodiment of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0062] Attached Figure 1 This is a flow chart of an image stabilization processing method provided in an embodiment of the present application. The method is applied to a vehicle equipped with an AR-HUD system, and is specifically applied to the AR-HUD system of the vehicle. The AR-HUD system application implements the AR-HUD. The method includes the following steps:
[0063] S110, obtaining a target pitch angle change amount and a vehicle acceleration of the vehicle at a first moment.
[0064] First, let's introduce the pitch angle of the vehicle. The pitch angle of the vehicle refers to the angle between the front axle and the rear axle of the vehicle. For the convenience of description, when the front axle of the vehicle is lifted upward relative to the rear axle, the pitch angle is positive, and when the front axle of the vehicle is tilted downward relative to the rear axle, the pitch angle is negative. Frequent changes in the pitch angle of the vehicle are the key factor causing the AR-HUD display to jitter. Therefore, adjusting the pitch angle of the vehicle to avoid frequent changes in the pitch angle of the vehicle can optimize the AR-HUD display, which is crucial to improving the driver's visual perception and safety.
[0065] The target pitch angle change is used to describe the change in the pitch angle (θ1) of the vehicle at the first moment (T1, i.e. the current moment) relative to the pitch angle (θ2) at the second moment (T2), for example, the target pitch angle change Δθ = θ1-θ2. Among them, T1 is earlier than T2. The AR-HUD system can improve the display of the AR-HUD by correcting the target pitch angle change.
[0066] In an embodiment of the present application, the AR-HUD system can obtain the IMU data measured by the IMU in real time, and obtain the pitch angle of the vehicle at the current moment and the target pitch angle change of the vehicle at the current moment according to the IMU data. Among them, the IMU data includes acceleration data and angular velocity data (i.e., gyroscope data). The acceleration data is measured and obtained using the accelerometer in the IMU, and the angular velocity data is measured and obtained using the gyroscope sensor in the IMU. The operating frequency of the IMU is usually above 100Hz, so the IMU data of the vehicle can be accurately obtained in real time using the IMU, and the IMU data can provide basic motion parameters, and the stability of the IMU can be maintained even in the case of laser radar interference, thereby further improving the accuracy of the IMU measurement.
[0067] In an embodiment of the present application, the AR-HUD system can obtain the pitch angle of the vehicle based on the acceleration data and the angular velocity data. Exemplarily, the AR-HUD system determines the acceleration component of the acceleration data in the direction perpendicular to the ground; integrating the angular velocity data around the Y-axis (perpendicular to the vehicle's forward direction) can obtain the rotation angle of the vehicle around the Y-axis, that is, the pitch angle of the vehicle; and using a fusion algorithm (such as a Kalman filter) to obtain the pitch angle of the vehicle. In this way, an accurate and stable pitch angle of the vehicle can be obtained. Then the AR-HUD system obtains the target pitch angle change based on the difference between the pitch angle at the current moment and the pitch angle at the second moment.
[0068] Vehicle acceleration is used to describe the change in the vehicle's speed at the current moment relative to the speed at the second moment. When the vehicle is shaken by human operation, the absolute value of the vehicle acceleration will be relatively large. Therefore, the AR-HUD system can determine whether the vehicle's shaking includes human operation shaking through the vehicle acceleration.
[0069] In an embodiment of the present application, the AR-HUD system first obtains the speed information of the vehicle, wherein the speed information of the vehicle is used to describe the speed of the vehicle and the time when the speed is collected, including the vehicle speed information and / or wheel speed (i.e., wheel speed) information of the vehicle. The speed information of the vehicle can be collected by sensors such as a vehicle-mounted speedometer and a driving recorder, and the wheel speed information of the vehicle can be collected by a Hall-type wheel speed sensor installed on the vehicle. AR-HUD determines the vehicle acceleration based on the speed information of the system vehicle. The vehicle speed obtained by the AR-HUD system at T1 is V1, and the vehicle speed obtained at T2 is V2, then the vehicle acceleration a is:
[0070]
[0071] Among them, Δt = T2 - T1. It can be understood that if it is to be accurately determined whether there is human-operated vibration during the vehicle's driving, the vehicle acceleration should be instantaneous acceleration, that is, T2 is infinitely close to T1.
[0072] S120: Determine, using the vehicle acceleration, whether the vehicle vibration includes human-operated vibration.
[0073] In an embodiment of the present application, the AR-HUD system determines whether the vehicle acceleration meets a preset condition. If the preset condition is met, it is determined that the vehicle vibration includes human-operated vibration.
[0074] The preset condition is set by a person skilled in the art as required to determine whether the vibration of the vehicle includes the vibration caused by human operation. In one example, the preset condition may be that the preset acceleration does not belong to the preset acceleration interval [a min ,a max ], and a min <a max That is, if the preset acceleration a satisfies a<a min , or a>a max , determine that the vehicle's shaking includes human-operated shaking, and determine that the vehicle's shaking includes human-operated shaking. It can be understood that if a min ≤a≤a max , it is determined that the vibration of the vehicle does not include human-operated vibration. The preset acceleration range may be a range set by those skilled in the art based on experience.
[0075] In another example, since human-operated jitter includes multiple types, such as jitter caused by sudden acceleration, jitter caused by sudden deceleration, and jitter caused by starting from a standstill, etc., the acceleration corresponding to each type of human-operated jitter is different. In order to improve the accuracy of determining whether the vehicle jitter includes human-operated jitter, the preset conditions in the AR-HUD system include a first preset condition, a second preset condition, and a third preset condition. Among them, the first preset condition indicates that the vehicle acceleration belongs to the first acceleration interval corresponding to sudden deceleration [a min1 ,a max1 The second preset condition indicates that the vehicle acceleration belongs to the second acceleration interval corresponding to the rapid acceleration [a min2 ,a max2 The third preset condition indicates that the vehicle acceleration belongs to the third acceleration interval corresponding to starting from a standstill [a min3 ,a max3 ]. If the vehicle's acceleration a is at [a min1 ,a max1 ], or in [a min2 ,a max2 ], or in [a min3 ,a max3 ], it is determined that the vehicle's vibration includes human-operated vibration. That is, when the vehicle acceleration meets any one of the first preset condition, the second preset condition, and the third preset condition, it is determined that the vehicle's vibration includes human-operated vibration. Thus, by setting the acceleration interval corresponding to each human-operated vibration, the determination accuracy is improved.
[0076] However, since the acceleration caused by sudden braking, sudden acceleration, and starting from a standstill is affected by many factors, such as the friction coefficient between the wheels and the ground, the weight of the vehicle, etc., the preset acceleration range set based on experience is often difficult to meet the design requirements. In view of this, the AR-HUD system uses the fact that there is only human operation jitter when driving on a smooth road, but no pothole jitter, to perform pre-operation and obtain the acceleration range.
[0077] Specifically, the AR-HUD system presets a first speed interval [Vmin1 (first speed value), Vmax1 (second speed value)], where Vmin1 < Vmax1. Vmin1 and Vmax1 are speed values set by technicians in this field as needed, for example, Vmin1 = 20km / h, Vmax1 = 80km / h. On a smooth road, based on the first speed interval, pre-operations such as sudden deceleration, sudden acceleration, starting at rest, uniform deceleration, uniform acceleration, slow acceleration when driving at a uniform speed, and slow deceleration are performed to obtain the acceleration interval of the vehicle during human-operated jitters such as sudden acceleration, sudden deceleration, and starting at rest.
[0078] Among them, AR-HUD system: sudden deceleration pre-operation refers to the vehicle speed being reduced from Vmax1 to close to Vmin1 within a preset time, and sudden acceleration pre-operation refers to the vehicle speed being accelerated from Vmin1 to close to Vmax1 within a preset time. The pre-operation for starting at rest refers to the vehicle accelerating from a stationary state to a stable speed (i.e., Vd, for example, Vd is Vmin1, the intermediate speed (Vmin1+Vmax1) / 2 or Vmax1) within a preset time. The uniform deceleration pre-operation refers to reducing the speed of the vehicle from Vd to Vmin1 at a constant deceleration. The uniform acceleration pre-operation refers to increasing the speed of the vehicle from Vd to Vmax1 at a constant acceleration. The slow acceleration and slow deceleration pre-operations during uniform speed driving refer to a small range of acceleration and deceleration operations under Vd, a slow deceleration operation that reduces the speed of the vehicle from Vd+α to Vd-α within a preset time, or a slow acceleration operation that increases the speed of the vehicle from Vd-α to Vd+α within a preset time. Wherein α is a speed value set by technicians in this field as needed. Among them, the preset time is a time greater than or equal to that set by technical personnel in this field as needed, for example, the preset time is 2s, 5s, 10s, etc., which is not specifically limited in the embodiments of the present application.
[0079] The AR-HUD system obtains the magnitude of various pre-operation target accelerations. The AR-HUD system analyzes the maximum and minimum values of the vehicle's target acceleration under each pre-operation, thereby determining the acceleration range of the vehicle in each pre-operation, specifically the first acceleration range corresponding to the rapid deceleration operation [a min1 ,a max1 ], the second acceleration interval corresponding to the rapid acceleration operation [a min2 ,a max2 ], the third acceleration range corresponding to starting from rest [a min3 ,a max3 ]. Thus, by pre-operation in advance, the preset conditions can meet the needs of the current vehicle, further improving the determination accuracy.
[0080] Furthermore, considering that there may be recognition errors when performing a pre-operation, the AR-HUD can perform N (an integer greater than 1) repeated operations for each pre-operation, and the AR-HUD system obtains the maximum and minimum values of the vehicle acceleration under each pre-operation, and determines the acceleration range of the vehicle in each pre-operation, thereby further improving the determination accuracy.
[0081] In addition, the AR-HUD system can also use the target pitch angle change of the vehicle to further verify whether the vehicle's jitter includes human-operated jitter. It can be understood that if the vehicle's jitter includes human-operated jitter, the vehicle's pitch angle must have a pitch angle change. To this end, using the target pitch angle change to further verify whether the vehicle's jitter includes human-operated jitter helps to improve the determination accuracy. Specifically, the AR-HUD system determines the target pitch angle change and presets a pitch angle change threshold. If the determined target pitch angle change is greater than the pitch angle change rate threshold, it is determined that the vehicle's jitter includes human-operated jitter. Otherwise, it is determined that the vehicle's jitter does not include human-operated jitter.
[0082] Among them, the pitch angle change threshold value can be a value set by those skilled in the art based on experience. In addition, the pitch angle change threshold value can also be obtained based on the first speed interval described above, by performing pre-operations such as rapid deceleration, rapid acceleration, starting from a standstill, uniform deceleration, uniform acceleration, slow acceleration and slow deceleration during uniform driving on a stable road section. At this time, the pitch angle change threshold value obtained meets the needs of the current vehicle and can improve the determination accuracy. In addition, the pitch angle change threshold value can also be determined by other methods, which are not specifically limited in the embodiments of the present application.
[0083] Furthermore, in order to determine whether the vehicle has experienced human-induced operational jitter, in addition to considering the pitch angle change mentioned above, the roll angle change (the change in the roll angle at the first moment relative to the roll angle at the second moment) and the yaw angle change (the change in the yaw angle at the first moment relative to the yaw angle at the second moment) may also be comprehensively considered. This is because when there is human-induced operational jitter such as a sharp turn, there will be a measurement error in the instantaneous acceleration of the vehicle speed information. Therefore, whether a sharp turn or other human-induced operational jitter has occurred may be determined based on the roll angle and yaw angle.
[0084] During a sharp turn, the roll angle changes significantly because the vehicle needs to move laterally around its axis. Monitoring the change in the roll angle can determine whether the vehicle is making a sharp turn. If the roll angle changes significantly in a short period of time, it may be caused by human operation. During a sharp turn, the yaw angle changes significantly. By monitoring the change in the yaw angle, it can be determined whether the vehicle is making a rapid directional adjustment. If the yaw angle changes significantly in a short period of time, it may be caused by human operation.
[0085] Relying solely on the change in roll angle or yaw angle may not be enough to accurately determine whether it is a human-operated jitter, because certain natural factors (such as uneven road surface) may also cause changes in these angles. Therefore, the change in roll angle and yaw angle should be considered comprehensively. If both change significantly at the same time, and this change is inconsistent with normal driving behavior (the change in roll angle exceeds the roll angle change threshold, and the change in yaw angle exceeds the yaw angle change threshold), it can be considered that the vehicle has undergone human-operated jitter. Specifically, in practical applications, reasonable thresholds need to be set for changes in roll angle and yaw angle. These thresholds can be adjusted according to factors such as vehicle type and driving environment. If the change in roll angle or yaw angle exceeds the set threshold, the alarm system is triggered to prompt the driver or vehicle management system to pay attention to possible human-operated jitter.
[0086] The embodiment of the present application may also determine whether the vibration of the vehicle includes human-operated vibration in other ways, which is not specifically limited in the embodiment of the present application.
[0087] S130: If the shaking of the vehicle includes human-operated shaking, filter the first pitch angle change amount from the target pitch angle change amount to obtain a second pitch angle change amount.
[0088] At the current moment, the target pitch angle change may include a first pitch angle change caused by human operation jitter (the change in the pitch angle caused by human operation jitter at the first moment relative to the pitch angle caused by human operation jitter at the second moment). If the AR-HUD system determines that the vehicle's jitter includes human operation jitter, it filters out the human operation jitter to eliminate the impact of human operation on the reverse compensation pitch angle. Specifically, the AR-HUD system filters the first pitch angle change from the target pitch angle change to obtain the second pitch angle change. Since human operation jitter is generally high-frequency jitter, the AR-HUD system can use the Kalman filtering method to filter the first pitch angle change from the target pitch angle change to obtain the second pitch angle change. Kalman filtering uses a recursive method to solve the linear filtering problem, which can effectively reduce the accumulation of errors when processing dynamic data such as target pitch angle change.
[0089] In addition, the AR-HUD system may also use other methods to filter out the first pitch angle change from the target pitch angle change, for example, using a sliding mode observer to filter out the first pitch angle change from the target pitch angle change, which is not specifically limited in the embodiments of the present application.
[0090] Since human operation jitter includes operation jitter such as sudden acceleration, sudden deceleration or starting from a standstill, the pitch angle change corresponding to each type of human operation jitter may be different. If the first pitch angle change only considers one of them for filtering, the second pitch angle change obtained after filtering has low accuracy. In view of this, AR-HUD obtains the pitch angle change corresponding to sudden deceleration, sudden deceleration, and starting from a standstill, among which the first pitch angle change includes the pitch angle change corresponding to sudden deceleration, the pitch angle change corresponding to sudden acceleration, and the pitch angle change corresponding to starting from a standstill.
[0091] If the vehicle's shaking includes human-operated shaking, and the human-operated shaking is caused by sudden deceleration, the pitch angle change corresponding to the sudden deceleration is filtered out from the target pitch angle change. If the human-operated shaking is caused by sudden acceleration, the pitch angle change corresponding to the sudden acceleration is filtered out from the target pitch angle change. If the human-operated shaking is caused by starting from a standstill, the pitch angle change corresponding to starting from a standstill is filtered out from the target pitch angle change. Thus, different first pitch angle changes are filtered out according to different types of human-operated shaking, and the second pitch angle change obtained is more accurate.
[0092] Exemplarily, a first sub-pitch angle change corresponding to a first acceleration interval, a second sub-pitch angle change corresponding to a second acceleration interval, and a third sub-pitch angle change corresponding to a third acceleration interval are determined. The first sub-pitch angle change is greater than the third sub-pitch angle change, and the second sub-pitch angle change is greater than the third sub-pitch angle change. If the vehicle acceleration satisfies the first acceleration interval, the first sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change; if the vehicle acceleration satisfies the second acceleration interval, the second sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change; if the vehicle acceleration satisfies the third acceleration interval, the third sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change.
[0093] S140: If the second pitch angle variation is greater than the first variation threshold, reversely compensate the second pitch angle variation to obtain a corrected pitch angle variation.
[0094] The first change threshold refers to a threshold set by technicians in this field as needed to determine whether the vehicle has pothole jitters, for example, 5°. If the second pitch angle change is greater than the first change threshold, it is determined that the vehicle's jitter includes pothole jitters, and the second pitch angle change is reversely compensated to obtain the corrected pitch angle change of the vehicle at the first moment. That is, a compensation value is set, which is positively correlated with the second pitch angle change, and the second pitch angle change is offset by the compensation value, thereby avoiding the impact of pothole jitters on the AR-HUD display.
[0095] S150: Determine the AR-HUD display image by using the corrected pitch angle variation.
[0096] The pitch angle of the vehicle at the current moment is corrected by using the corrected pitch angle change, and the rotation coordinates are calculated using the corrected pitch angle and the current road inclination angle, where the rotation coordinates are used to describe the position and posture of the vehicle relative to the road surface. The AR-HUD system adjusts the display angle of the vehicle based on the rotation coordinates, thereby ensuring that the virtual image can be accurately superimposed on the actual road conditions to obtain an accurate AR-HUD display image, which is displayed on the AR-HUD display screen. Since the corrected pitch angle eliminates the impact of pothole jitter and human operation jitter, it helps to reduce the jitter of the AR-HUD display screen.
[0097] Furthermore, since the vehicle still has road noise jitter during driving, the AR-HUD system first processes the vehicle jitter through a low-pass filter to filter out the road noise jitter. That is, the third pitch angle change generated by the road noise jitter is filtered out from the target pitch angle change using a low-pass filter to obtain the initial corrected pitch angle change. At this time, for step S130, if the vehicle jitter includes manual operation jitter, the first pitch angle change is filtered out from the initial corrected pitch angle change. In this way, the influence of road noise jitter on the AR-HUD display is eliminated.
[0098] Optionally, after executing step S140, step S160 may be executed, and then step S150 may be executed.
[0099] S160: If the corrected pitch angle variation is greater than a third variation threshold, correct the pitch angle variation based on a smoothing algorithm and superimposed reverse compensation processing.
[0100] The third change threshold is used to determine whether to offset the second pitch angle change after reverse compensation for the second pitch angle change, so as to avoid the influence of the second pitch angle change on the AR-HUD display. Among them, the third change threshold is a value less than the first change threshold. If the corrected pitch angle change is greater than the third change threshold, it means that the second pitch angle change is not completely offset, and the corrected pitch angle change will also cause jitter in the AR-HUD display. At this time, the AR-HUD system also needs to use a smoothing algorithm to reverse compensate for the corrected pitch angle change.
[0101] Among them, the purpose of the smoothing algorithm is to reduce the noise of the corrected pitch angle change, improve the signal-to-noise ratio, and make the corrected pitch angle change smoother and continuous. In the embodiment of the present application, the smoothing algorithm can be a moving average smoothing method or a convolution smoothing method, etc. The embodiment of the present application is not specifically limited. In addition, since the smoothing algorithm processes the corrected pitch angle change, it will change the changing trend of the corrected pitch angle change. Therefore, the reverse compensation can offset the change in the corrected pitch angle change caused by the smoothing algorithm. That is, by superimposing the reverse compensation on the smoothing algorithm to process the corrected pitch angle change, the sudden changes in the AR-HUD display screen can be reduced, which is beneficial to improve the stability of the AR-HUD display screen and provide the driver with a more stable and smooth driving experience.
[0102] In one example, the AR-HUD system can use a sliding window method superimposed with a Kalman filter to process the compensated pitch angle to correct the pitch angle change. Specifically, the AR-HUD system collects a corrected pitch angle change sequence in real time. The corrected pitch angle change sequence includes the corrected pitch angle changes at the current moment and at multiple previous moments. For example, the corrected pitch angle change sequence is {x n , x n -1, ..., x2, x1}, where x n is the modified pitch angle change at time Tn, x n-1 is the modified pitch angle change at time Tn-1, ...x2 is the modified pitch angle change at time T2, and x1 is the modified pitch angle change at time T1 (current time). Tn, Tn-1...T2 are moments before the current moment. It can be understood that the modified pitch angle change sequence can also be displayed in other ways, which is not specifically limited in the embodiments of the present application. n is an integer greater than 1.
[0103] AR-HUD sets the size of the sliding window. For example, the size of a sliding window includes N data points (the selection of N depends on the characteristics of the data and the required degree of smoothness. N is an integer greater than or equal to 1, and N is less than n), where 1 data point represents a corrected pitch angle change. The sliding window slides on the corrected pitch angle sequence, processing N data points in the window each time. Among them, specific processing methods include averaging, median or other statistical methods to reduce the impact of noise and outliers. After processing the data of a window, the window slides forward one data point.
[0104] The AR-HUD system sets the initial state and initial error covariance matrix of the Kalman filter based on the processing results of the first sliding window. Next, the AR-HUD system predicts the filter prediction value at the current moment using the filter estimation value at the previous moment according to the state transfer equation. And predict the prediction error covariance matrix at the current moment. Then, the AR-HUD system updates the filter prediction value and prediction error covariance matrix for the processing results of each sliding window. The Kalman gain corresponding to each sliding window is calculated, and the optimal state estimation value corresponding to each sliding window is obtained by combining the filter prediction value and the prediction error covariance matrix, and the error covariance matrix is updated. The two stages of prediction and update are repeated. Through this method, relatively smooth pitch angle data can be provided, and noise and interference can be further reduced, thereby improving the accuracy and smoothness of state estimation, thereby providing a more stable and smooth driving experience for the AR-HUD system.
[0105] In another example, the AR-HUD system can use interpolation to digitize the AR-HUD display interface to avoid the impact of the corrected pitch angle change on the AR-HUD display interface. Specifically, the AR-HUD system collects the corrected pitch angle change in real time, performs preprocessing such as denoising and calibration on the corrected pitch angle change, and obtains an accurate corrected pitch angle change sequence. The AR-HUD system selects key data points as interpolation points from the corrected pitch angle change sequence, where the interpolation point refers to the data point that reflects the overall change trend of the data, such as the interpolation point is a peak value, an estimate, and a turning point. For example, if the corrected pitch angle change sequence is [5°, 6°, 7°, 8°, 10°, 12°, 11°, 9°], 5°, 8°, 12°, and 9° can be selected as interpolation points because they represent significant changes in the data. By accurately selecting interpolation points, the accuracy and smoothness of the interpolation results can be guaranteed.
[0106] Based on a preset interpolation method (e.g., a linear interpolation method or a spline interpolation method), the data values between the interpolation points are determined to generate a smooth sequence of corrected pitch angle changes. For example, the AR-HUD system uses a linear interpolation method to calculate the data values between the interpolation points using a linear equation based on the data values between the two interpolation points. For example, if 5° and 8° are selected as interpolation points, and linear interpolation is used to calculate the values of 6° and 7° between them, the following formula can be used:
[0107]
[0108] Among them, x1=5, y1=5, x2=8, y2=8. Substituting 6° and 7° into x respectively, the corresponding y values can be calculated, that is, the value corresponding to 6° is approximately 5.75°, and the value corresponding to 7° is approximately 6.5°.
[0109] The interpolation method can be used to smoothly correct the pitch angle variation sequence and obtain a smoother processing result. The AR-HUD system adjusts the pitch angle of the AR-HUD display image according to the smoothed corrected pitch angle variation sequence, such as changing the angle, position or zoom of the AR-HUD display image, to ensure that the displayed content is consistent with the actual driving environment. That is, through the interpolation method, the AR-HUD display screen jitter caused by the failure to fully offset the second pitch angle variation can be avoided, further improving the accuracy and smoothness of the displayed AR-HUD image, effectively improving the AR-HUD display screen jitter, and providing the driver with a more stable, intuitive and safe driving experience.
[0110] In addition, the embodiments of the present application may also perform reverse smooth compensation on the compensated pitch angle in other ways, which are not specifically limited in the present application.
[0111] Further, in an embodiment of the present application, the AR-HUD system can cache the angle change of the vehicle pitch angle of the target vehicle within a period of time (preset time period) and calculate the average value, which is the pitch angle reference value for this period of time. In a flat road section, that is, before entering / after exiting a pit, this data can represent the road surface angle of the current target vehicle. It should be noted that if the current vehicle is driving into a pit, in a pit, or before exiting a pit, the sliding window data is not updated. The AR-HUD system inputs the preset vehicle pitch angle into the Kalman filter for processing, thereby obtaining accurate vehicle posture data based on the vehicle posture input by the target vehicle, predicting the next frame of data, and obtaining the predicted vehicle pitch angle. The AR-HUD system combines the above-mentioned sliding window method, the predicted vehicle pitch angle, and the road surface angle of the target vehicle before entering the pit to obtain the actual pitch angle of the target vehicle relative to the road surface. If the actual pitch angle is greater than the preset pitch angle threshold, the pitch angle is reversely smoothed and compensated. By providing accurate vehicle posture data and smooth image or information display, the AR-HUD system can further enhance the user's driving experience and safety.
[0112] Therefore, the embodiment of the present application obtains the target pitch angle change and vehicle acceleration of the vehicle at the first moment, and uses the vehicle acceleration to determine whether the vehicle's jitter includes human-operated jitter. If there is human-operated jitter, the pitch angle change corresponding to the human-operated jitter (referred to as the first pitch angle change) is filtered out from the target pitch angle change of the vehicle. In other words, the embodiment of the present application identifies human-operated jitter from the jitter of the vehicle, filters out the human-operated jitter, and only retains the pothole jitter, and performs reverse compensation pitch angle processing on the pothole jitter, thereby avoiding the influence of human-operated jitter on the AR-HUD display screen, improving the phenomenon of frequent and violent jumping of the AR-HUD display screen, and helping to improve the driver's visual perception and safety.
[0113] Furthermore, considering that during vehicle driving, the vehicle Figure 1 The image stabilization processing method shown in the figure will cause the AR-HUD system to have low processing efficiency. In view of the above problems, the present application embodiment provides another image stabilization processing method. In order to better illustrate another image stabilization processing method provided by the present application embodiment, the following is combined with the attached Figure 2 ~Attached Figure 3 The following is an explanation.
[0114] Attached Figure 2 This is a flow chart of another image stabilization processing method provided in an embodiment of the present application. The method is applied to a vehicle equipped with an AR-HUD, and is specifically applied to an AR-HUD system of the vehicle. The AR-HUD system application implements the AR-HUD. The method includes the following steps:
[0115] S210: Obtain the inclination angle of the current road.
[0116] The current road refers to the road on which the vehicle is traveling. The current road may be a flat road, an uphill road, or a downhill road, which is not specifically limited in the embodiments of the present application.
[0117] In the embodiment of the present application, the AR-HUD system can first obtain the inclination angle of the current road. Thus, the AR-HUD system adjusts the display angle and position of the AR-HUD display image according to the inclination angle of the current road and the posture data of the vehicle, thereby ensuring that the displayed information is always consistent with the driver's line of sight and the road status.
[0118] Among them, the AR-HUD system can obtain P times (P is an integer of 1, for example, P = 5) of pitch angle changes when the vehicle is started. If the P times of pitch angle changes fluctuate at threshold A and are all less than threshold A, the inclination angle of the current road is threshold A. Therefore, the AR-HUD system can quickly initialize and determine a road inclination angle, which helps to provide accurate driving assistance information to the driver.
[0119] S220: Obtain IMU data and vehicle speed information.
[0120] The specific acquisition method refers to the method described in step S110 and will not be discussed here.
[0121] In an embodiment of the present application, the AR-HUD system can determine the pitch angle of the vehicle based on the IMU data at the current moment. In addition, the AR-HUD system can also determine the roll angle of the vehicle and the yaw angle of the vehicle based on the IMU data at the current moment. In addition, the AR-HUD system can also determine the target pitch angle change, roll angle change, and yaw angle change of the vehicle based on the IMU data at the current moment.
[0122] S230: Determine the road section type of the vehicle's driving section according to the IMU data of the vehicle.
[0123] The AR-HUD system can determine the type of the road section on which the vehicle is traveling based on the change in the vehicle's pitch angle and the change in gravity acceleration, where the change in gravity acceleration of the vehicle refers to the change in gravity acceleration at a first moment relative to gravity acceleration at a second moment.
[0124] Specifically, if the AR-HUD system detects that the target pitch angle change of the vehicle is less than or equal to the second change threshold, and the gravity acceleration change of the vehicle is not less than the gravity acceleration change threshold, the driving section is determined to be a smooth section. It should be noted that a smooth section refers to a section where the fluctuation amplitude value is less than the fluctuation amplitude threshold. If the AR-HUD system detects that the target pitch angle change of the vehicle is greater than the second change threshold and the gravity acceleration change of the vehicle is less than the gravity acceleration change threshold, the driving section is determined to be in the pit entry stage.
[0125] Example description: Attached Figure 3 A schematic diagram of a vehicle driving on a current road provided in an embodiment of the present application, wherein section ① is a smooth section, section ② is a section for entering a pit, section ③ is a section for passing a pit, and section ④ is a smooth section after exiting a pit. Figure 3 (a) in the figure shows that the pothole on the road is relatively large, and the vehicle needs to drive in the pothole to pass through section ③. Figure 3 (b) in the figure shows that the pothole on the road is relatively small and the vehicle is not driving in the pothole.
[0126] When the vehicle is on a smooth road, the AR-HUD system assumes that there is no pothole jitter by default. If the vehicle is jittering, it may be caused by human operation. If the vehicle is driving on a road with a pit, the pitch angle change caused by the pothole jitter is much greater than the pitch angle change caused by human operation jitter. Therefore, when the vehicle is driving on a road with a pit, it is assumed that there is no human operation jitter, only pothole jitter. For example, Figure 3 The AR-HUD system defaults to human-operated jitter and pothole jitter on section ③, pothole jitter on section ② and no human-operated jitter, and human-operated jitter on sections ① and ④ but no pothole jitter.
[0127] S240: If the vehicle is traveling on a smooth road or a pothole-crossing road, determine whether the vehicle's vibration includes human-operated vibration based on the vehicle's speed information. If the vehicle's vibration includes human-operated vibration, execute step S130. If the vehicle's vibration does not include human-operated vibration, execute S250.
[0128] In the embodiment of the present application, the AR-HUD system first uses the vehicle speed information to determine the vehicle acceleration at the first moment, and uses whether the vehicle acceleration meets the preset conditions to determine whether the vehicle vibration includes human-operated vibration. The specific judgment method is referred to step S120, which is not discussed here.
[0129] In addition, the AR-HUD system can determine whether the vehicle is traveling based on the vehicle's speed information. Specifically, the AR-HUD presets a driving threshold value V0. If the vehicle's speed is greater than V0, it can be determined that the vehicle is traveling. If the vehicle's speed is not greater than V0, it can be determined that the vehicle is not traveling. For example, V0 is 0.5km / h or 1km / h, etc., which is not specifically limited in the embodiments of the present application. If the vehicle is not in a driving state, there is no jitter in the vehicle. At this time, in order to improve the efficiency of the image anti-shake processing, there is no need for subsequent processing of the jitter.
[0130] S250 , reversely compensate the target pitch angle variation to obtain a corrected pitch angle variation.
[0131] In the embodiment of the present application, S150 is executed after executing S250, or S160 is executed, which will not be discussed here.
[0132] The embodiment of the present application determines the driving section of the vehicle through the vehicle's posture data, and processes the jitter of the vehicle in different sections differently, which improves the processing effect on the one hand, and can accurately process the jitter of the vehicle on the other hand. Finally, the AR-HUD system uses reverse smoothing compensation processing on the compensated pitch angle again, which can make the displayed image smoother and more conducive to improving the driver's visual perception and safety.
[0133] In addition, the embodiment of the present application also provides another image stabilization processing method, which can be customized to meet image stabilization requirements.
[0134] Attached Figure 4 This is another flow chart of an image stabilization processing method provided in an embodiment of the present application. The method is applied to a vehicle equipped with an AR-HUD, and is specifically applied to an AR-HUD system of the vehicle. The AR-HUD system application implements the AR-HUD. The method includes the following steps:
[0135] S410 , testing various road conditions based on the second speed range, and collecting jitter data after the test.
[0136] Among them, the second speed interval is [Vmin2, Vmax2], wherein Vmin2<Vmax2, for example, Vmin2=20km / h, Vmax2=80km / h.
[0137] The AR-HUD system obtains the jitter data of the vehicle in the second speed range when testing various road conditions, including speed bumps, railroad tracks, pits, gravel asphalt roads and other bumpy sections. The jitter data after the test includes the jitter frequency at the driving position of the vehicle and the jitter amplitude at the pitch angle.
[0138] AR-HUD combines the jitter data after the test to divide the road conditions that the vehicle passes through into large-amplitude jitter conditions, medium-amplitude jitter conditions and small-amplitude jitter conditions.
[0139] A road condition with severe shaking refers to a road condition in which the pitch angle shaking amplitude is greater than a first amplitude threshold, and the shaking frequency is less than a first frequency threshold. For example, when a vehicle passes through a pothole, if the diameter of the pothole is greater than the diameter of the vehicle's front wheels, the pitch angle shaking amplitude is greater than the first amplitude threshold, and the shaking frequency is less than the first frequency threshold. This road condition is a road condition with severe shaking.
[0140] Medium-amplitude jitter road conditions refer to road conditions in which the jitter amplitude of the pitch angle is greater than the second amplitude threshold and less than or equal to the first amplitude threshold, and the jitter frequency is not less than the first frequency threshold and less than the second frequency threshold. For example, when a vehicle passes over a speed bump, train tracks, etc., or the diameter of the pit is not greater than the diameter of the vehicle's front wheels, the jitter amplitude of the pitch angle generated is greater than the second amplitude threshold and less than or equal to the first amplitude threshold, and the jitter frequency is not less than the first frequency threshold and greater than the second frequency threshold. This road condition can be a medium-amplitude jitter road condition.
[0141] A road condition with small vibration refers to a road condition in which the vibration amplitude of the pitch angle is less than or equal to the second amplitude threshold, and the vibration frequency is greater than or equal to the second frequency threshold. For example, when a vehicle passes through a gravel asphalt road that has not been flattened by a roller, the vibration amplitude of the pitch angle generated is less than or equal to the second amplitude threshold, and the vibration frequency is greater than or equal to the second frequency threshold. This road condition is a road condition with small vibration.
[0142] In the embodiments of the present application, the process may be performed multiple times, and the specific number of times may be determined by those skilled in the art according to the needs.
[0143] S420: Obtain a low-pass filter corresponding to each road condition.
[0144] Among them, the large-scale shaking road condition corresponds to the first low-pass filter, the medium-scale shaking road condition corresponds to the second low-pass filter, and the small-scale shaking road condition corresponds to the third low-pass filter. The first low-pass filter corresponds to the shaking frequency of the large-scale shaking road condition and is used to filter the road noise shaking under the large shaking. The second low-pass filter corresponds to the shaking frequency of the medium-scale shaking road condition and is used to filter the road noise shaking under the medium-scale shaking. The third low-pass filter corresponds to the shaking frequency of the small-scale shaking road condition and is used to filter the road noise shaking under the small shaking. Therefore, the third pitch angle change generated by the road noise shaking is filtered out from the target pitch angle change by using the low-pass filter, thereby eliminating the influence of the road noise shaking on the pothole shaking.
[0145] S430: Obtain a target vibration frequency and a target pitch angle vibration amplitude when the vehicle is traveling on the current road.
[0146] S440: Determine the current road condition.
[0147] The AR-HUD system determines whether the current road condition is a large-amplitude shaking road condition, a medium-amplitude shaking road condition, or a small-amplitude shaking road condition based on the target shaking frequency and the target pitch angle shaking amplitude and the classification criteria for pothole sections.
[0148] S450: Filter using a low-pass filter corresponding to the road condition.
[0149] In addition, the AR-HUD system uses a low-pass filter corresponding to the road condition to determine whether the road condition belongs to the target road condition. The target road condition is the road condition for vehicle anti-shake processing set by technicians in the field as needed. The target road condition can be at least one of a small-amplitude shaking road condition, a medium-amplitude shaking road condition, and a large-amplitude shaking road condition. If the road condition belongs to the target road condition, the AR-HUD system executes the attached Figure 1 or attached Figure 2 The image stabilization processing method shown in FIG. will not be discussed here.
[0150] Therefore, by classifying the pothole sections, the embodiments of the present application allow technicians in this field to handle the large-amplitude shaking road conditions, medium-amplitude shaking road conditions, or all pothole conditions, so it has strong flexibility and practicality.
[0151] In addition, the embodiment of the present application also provides an image stabilization processing device, which is applied to a vehicle, and specifically to an AR-HUD system of the vehicle. Figure 5 An image stabilization processing device is provided for an embodiment of the present application. The device 500 includes:
[0152] An acquisition unit 501 is used to acquire a target pitch angle change amount and a vehicle acceleration of the vehicle at a first moment, wherein the target pitch angle change amount indicates a change amount of the pitch angle at the first moment relative to the pitch angle at a second moment;
[0153] The determining unit 502 is configured to determine that the vibration of the vehicle includes human-operated vibration if the vehicle acceleration satisfies a preset condition; filter out the first pitch angle variation from the target pitch angle variation to obtain a second pitch angle variation; the first pitch angle variation indicates a variation of the pitch angle caused by human-operated vibration at a first moment relative to the pitch angle caused by human-operated vibration at a second moment;
[0154] The compensation display unit 503 is used to reversely compensate the second pitch angle change if the second pitch angle change is greater than the first change threshold, so as to obtain a corrected pitch angle change of the vehicle at the first moment; and determine the AR-HUD display image using the corrected pitch angle change.
[0155] Optionally, the preset condition includes a first preset condition, a second preset condition and a third preset condition, the first preset condition indicating that the vehicle acceleration belongs to a first acceleration interval corresponding to sudden deceleration, the second preset condition indicating that the vehicle acceleration belongs to a second acceleration interval corresponding to sudden acceleration, and the third preset condition indicating that the vehicle acceleration belongs to a third acceleration interval corresponding to starting from a stationary state;
[0156] If the vehicle acceleration satisfies a preset condition, determining that the vehicle vibration includes the human-operated vibration includes: if the vehicle acceleration satisfies the first preset condition, the second preset condition or the third preset condition, determining that the vehicle vibration includes the human-operated vibration.
[0157] The device 500 further includes a pre-operation unit 504, which is used to: based on the first speed interval, perform a pre-operation of rapid deceleration, a pre-operation of rapid acceleration, and a pre-operation of starting at rest on a stable road section, and obtain a first acceleration interval, a second acceleration interval, and a third acceleration interval; the stable road section indicates a road section where the fluctuation amplitude value is less than the fluctuation amplitude threshold;
[0158] The first speed interval is an interval from a first speed value to a second speed value, and the first speed value is less than the second speed value; the rapid acceleration pre-operation indicates an operation of accelerating the speed value of the vehicle from the first speed value to the second speed value at a preset time; the rapid deceleration pre-operation indicates an operation of decelerating the speed value of the vehicle from the second speed value to the first speed value at a preset time; the stationary start pre-operation indicates that the speed value of the vehicle is accelerated from a stationary state to a stable speed;
[0159] The compensation display unit 503 is specifically used to: determine a first sub-pitch angle change corresponding to a first acceleration interval, a second sub-pitch angle change corresponding to a second acceleration interval, and a third sub-pitch angle change corresponding to a third acceleration interval; if the vehicle acceleration satisfies the first acceleration interval, the first sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change; if the vehicle acceleration satisfies the second acceleration interval, the second sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change; if the vehicle acceleration satisfies the third acceleration interval, the third sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change.
[0160] The determining unit 502 is further configured to:
[0161] Obtain the vehicle's yaw angle change and roll angle change at a first moment; the yaw angle change indicates the change of the yaw angle at the first moment relative to the yaw angle at the second moment, and the roll angle change indicates the change of the roll angle at the first moment relative to the roll angle at the second moment; if the vehicle acceleration does not meet the preset conditions, determine whether the jitter includes human-operated jitter based on the yaw angle change and the roll angle change.
[0162] The determination unit 502 is also used for: if the vehicle is traveling on a road section passing through a pothole or a smooth road section at the first moment, and if the vehicle acceleration meets a preset condition, determining that the vehicle vibration includes human operation vibration; if the target pitch angle change is less than a second change threshold, and the vehicle's gravity acceleration change is less than the gravity acceleration change threshold, determining that the vehicle is traveling on a road section entering a pothole at the first moment; reversely compensating the target pitch angle change to obtain a corrected pitch angle change.
[0163] The compensation display unit 503 is specifically used for:
[0164] If the corrected pitch angle variation is greater than a third variation threshold, the pitch angle variation is corrected based on a smoothing algorithm and reverse compensation; and an AR-HUD display image is determined using the processed corrected pitch angle variation;
[0165] or,
[0166] The corrected pitch angle variation is preprocessed to obtain a corrected pitch angle variation sequence; a plurality of interpolation points are selected from the corrected pitch angle variation sequence; based on a preset interpolation method, data values between the plurality of interpolation points are determined to obtain a smooth corrected pitch angle variation sequence; and an AR-HUD display image is determined according to the smooth corrected pitch angle variation sequence.
[0167] The determining unit 502 is further configured to:
[0168] Filtering a third pitch angle variation caused by road noise jitter from the target pitch angle variation to obtain an initial corrected pitch angle variation; filtering the first pitch angle variation from the target pitch angle variation to obtain a second pitch angle variation, including: filtering the first pitch angle variation from the initial corrected pitch angle variation to obtain a second pitch angle variation;
[0169] and / or,
[0170] Based on the second speed range, a vehicle is used to test various road conditions that produce pothole jitters, and jitter data of each of the multiple road conditions is collected; the jitter data includes jitter frequency and pitch angle jitter amplitude; based on the jitter data of each road condition, the multiple road conditions are classified into large-amplitude jitter road conditions, medium-amplitude jitter road conditions, and small-amplitude jitter road conditions;
[0171] Among them, the large-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is greater than the first amplitude threshold, and the shaking frequency is less than the first frequency threshold; the medium-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is greater than the second amplitude threshold, and is less than or equal to the first amplitude threshold, and the shaking frequency is greater than or equal to the first frequency threshold, and is less than the second frequency threshold; the small-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is less than or equal to the second amplitude threshold, and the shaking frequency is greater than or equal to the second frequency threshold;
[0172] The acquisition unit 501 is specifically used for:
[0173] If the road condition corresponding to the driving section of the vehicle at the first moment belongs to the target road condition, the target pitch angle change and vehicle acceleration of the vehicle at the first moment are obtained; the target road condition is at least one of a large-amplitude shaking road condition, a medium-amplitude shaking road condition and a small-amplitude shaking road condition.
[0174] The embodiment of the present application provides an image anti-shake processing device, which obtains the target pitch angle change and vehicle acceleration of the vehicle at the first moment, and uses the vehicle acceleration to determine whether the vehicle's shaking includes human-operated shaking. If there is human-operated shaking, the pitch angle change corresponding to the human-operated shaking (referred to as the first pitch angle change) is filtered out from the target pitch angle change of the vehicle. In other words, the embodiment of the present application identifies human-operated shaking from the shaking of the vehicle, filters out the human-operated shaking, and only retains pothole shaking, and performs reverse compensation pitch angle processing on the pothole shaking, thereby avoiding the influence of human-operated shaking on the AR-HUD display screen, improving the phenomenon of frequent and violent jumping of the AR-HUD display screen, and helping to improve the driver's visual perception and safety.
[0175] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using a software program, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions in the embodiments of the present application are generated.
[0176] The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a magnetic disk, a magnetic tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.
[0177] The above are only preferred embodiments of the present application, and are not intended to limit the present application in any form. Although the present application has been disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the art can use the above disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present application without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still falls within the scope of protection of the technical solution of the present application.
Claims
1. An image stabilization processing method, characterized in that: The method comprises: Acquire a target pitch angle change amount and a vehicle acceleration of the vehicle at a first moment, wherein the target pitch angle change amount indicates a change amount of the pitch angle at the first moment relative to the pitch angle at a second moment; If the vehicle acceleration meets a preset condition, determining that the vehicle vibration includes human-operated vibration; filtering the first pitch angle variation from the target pitch angle variation to obtain a second pitch angle variation; the first pitch angle variation indicates a variation of the pitch angle caused by human-operated vibration at the first moment relative to the pitch angle caused by human-operated vibration at the second moment; If the second pitch angle change is greater than the first change threshold, the second pitch angle change is reversely compensated to obtain a corrected pitch angle change of the vehicle at the first moment; and the augmented reality head-up display AR-HUD display image is determined using the corrected pitch angle change.
2. The method according to claim 1, characterized in that: The preset conditions include a first preset condition, a second preset condition and a third preset condition, wherein the first preset condition indicates that the vehicle acceleration belongs to a first acceleration interval corresponding to a sudden deceleration, the second preset condition indicates that the vehicle acceleration belongs to a second acceleration interval corresponding to a sudden acceleration, and the third preset condition indicates that the vehicle acceleration belongs to a third acceleration interval corresponding to a stationary start; If the vehicle acceleration satisfies a preset condition, determining that the vehicle vibration includes the human-operated vibration includes: if the vehicle acceleration satisfies the first preset condition, the second preset condition or the third preset condition, determining that the vehicle vibration includes the human-operated vibration.
3. The method according to claim 2, characterized in that: The method further comprises: Based on the first speed interval, performing a pre-operation of sudden deceleration, a pre-operation of sudden acceleration and a pre-operation of starting at rest on a smooth road section, and acquiring the first acceleration interval, the second acceleration interval and the third acceleration interval; the smooth road section indicates a road section where the fluctuation amplitude value is less than the fluctuation amplitude threshold; Among them, the first speed interval is the interval from the first speed value to the second speed value, and the first speed value is less than the second speed value; the rapid acceleration pre-operation indicates an operation of accelerating the speed value of the vehicle from the first speed value to the second speed value within a preset time; the rapid deceleration pre-operation indicates an operation of decelerating the speed value of the vehicle from the second speed value to the first speed value within a preset time; the stationary starting pre-operation indicates that the speed value of the vehicle is accelerated from a stationary state to a stable speed.
4. The method according to claim 2, characterized in that: The method further comprises: Determine a first sub-pitch angle change amount corresponding to the first acceleration interval, a second sub-pitch angle change amount corresponding to the second acceleration interval, and a third sub-pitch angle change amount corresponding to the third acceleration interval; The filtering out the first pitch angle variation from the target pitch angle variation to obtain the second pitch angle variation includes: If the vehicle acceleration satisfies the first acceleration interval, filtering the first sub-pitch angle change from the target pitch angle change to obtain a second pitch angle change; If the vehicle acceleration satisfies the second acceleration interval, filtering the second sub-pitch angle change from the target pitch angle change to obtain a second pitch angle change; If the vehicle acceleration satisfies the third acceleration interval, the third sub-pitch angle change is filtered out from the target pitch angle change to obtain a second pitch angle change.
5. The method according to claim 2, characterized in that: The method further comprises: Acquire a yaw angle change and a roll angle change of the vehicle at the first moment; the yaw angle change indicates a change of the yaw angle at the first moment relative to the yaw angle at the second moment, and the roll angle change indicates a change of the roll angle at the first moment relative to the roll angle at the second moment; The method further comprises: If the vehicle acceleration does not meet the preset condition, it is determined whether the vibration includes the human-operated vibration according to the yaw angle change and the roll angle change.
6. The method according to claim 1, characterized in that: If the acceleration of the vehicle satisfies a preset condition, determining that the vibration of the vehicle includes human-operated vibration includes: If the driving section of the vehicle at the first moment is a pothole section or a smooth section, and if the acceleration of the vehicle meets a preset condition, determining that the vibration of the vehicle includes human-operated vibration; If the target pitch angle change is less than a second change threshold, and the vehicle's gravity acceleration change is less than the gravity acceleration change threshold, it is determined that the driving section of the vehicle at the first moment is a pit entry section; reversely compensate the target pitch angle change to obtain the corrected pitch angle change.
7. The method according to claim 1, characterized in that: The method of using the modified pitch angle variation to determine an augmented reality head-up display (AR-HUD) display image includes: If the modified pitch angle variation is greater than a third variation threshold, processing the modified pitch angle variation based on a smoothing algorithm and superimposing reverse compensation; and determining the AR-HUD display image using the processed modified pitch angle variation; or, Preprocessing the corrected pitch angle variation to obtain a corrected pitch angle variation sequence; Filtering a plurality of interpolation points from the modified pitch angle variation sequence; Based on a preset interpolation method, determining data values between the plurality of interpolation points to obtain a smoothed sequence of corrected pitch angle variation; The AR-HUD display image is determined according to the smoothed modified pitch angle variation sequence.
8. The method according to claim 1, characterized in that: If the vehicle acceleration satisfies a preset condition, before determining that the vehicle vibration includes human-operated vibration, the method further includes: filtering a third pitch angle variation caused by road noise jitter from the target pitch angle variation to obtain an initial corrected pitch angle variation; The filtering out the first pitch angle change amount from the target pitch angle change amount to obtain the second pitch angle change amount comprises: filtering out the first pitch angle change amount from the initial corrected pitch angle change amount to obtain the second pitch angle change amount; and / or, Based on the second speed range, using the vehicle to test multiple road conditions that produce pothole jitters, and collecting jitter data for each of the multiple road conditions; the jitter data includes jitter frequency and pitch angle jitter amplitude; Based on the jitter data of each road condition, the multiple road conditions are classified into a large-amplitude jitter road condition, a medium-amplitude jitter road condition and a small-amplitude jitter road condition; Among them, the large-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is greater than the first amplitude threshold, and the shaking frequency is less than the first frequency threshold; the medium-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is greater than the second amplitude threshold, and less than or equal to the first amplitude threshold, and the shaking frequency is greater than or equal to the first frequency threshold, and less than the second frequency threshold; the small-amplitude shaking road condition indicates the road condition in which the pitch angle shaking amplitude is less than or equal to the second amplitude threshold, and the shaking frequency is greater than or equal to the second frequency threshold; The step of obtaining a target pitch angle change amount and a vehicle acceleration of the vehicle at a first moment includes: If the road condition corresponding to the driving section of the vehicle at the first moment belongs to the target road condition, the target pitch angle change and vehicle acceleration of the vehicle at the first moment are obtained; the target road condition is at least one of the large-amplitude shaking road condition, medium-amplitude shaking road condition and small-amplitude shaking road condition.
9. An image stabilization processing device, characterized in that: The device comprises: an acquisition unit, configured to acquire a target pitch angle change amount of the vehicle at a first moment and a vehicle acceleration, wherein the target pitch angle change amount indicates a change amount of the pitch angle at the first moment relative to the pitch angle at a second moment; A determination unit, configured to determine that the jitter of the vehicle includes human-operated jitter if the vehicle acceleration satisfies a preset condition; filter out a first pitch angle change from the target pitch angle change to obtain a second pitch angle change; the first pitch angle change indicates a change in the pitch angle caused by human-operated jitter at the first moment relative to the pitch angle caused by human-operated jitter at the second moment; The compensation display unit is used to reversely compensate the second pitch angle change if the second pitch angle change is greater than the first change threshold, so as to obtain the corrected pitch angle change of the vehicle at the first moment; and use the corrected pitch angle change to determine the AR-HUD display image.
10. An electronic device, characterized in that: The electronic device comprises a memory and a processor; The memory is coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 8.
Citation Information
Cited By
Vehicle-mounted AR-HUD real-time alignment system and method based on multi-modal fusion
CN121632183A
